Composite image output apparatus and composite image delivery apparatus

ABSTRACT

A composite image output/delivery apparatus for ( 10 ) adding to a picked-up scene image a corresponding animation image and outputting/delivering a resulting composite image without adding a fixed animation image simply to the picked-up scene image. When an object image is extracted from the picked-up scene image received via an image input unit ( 17 ), any specified type of character image corresponding to the scene image is selected from an appropriate one of a character  1  “memory ( 12 ) e , a character  3  “cat” pose image memory ( 12   f ). The selected type of character image replaces the object image in the picked-up scene image or adds to the scene image to thereby provide and display a resulting composite image. That is, by replacing the object image with an animation image similar in pose to the object image or by adding the animation image to the scene image, a very delightful picked-up image with an animation is easily created without adding a fixed animation image simply to an image pickup framerabbit” pose image memory ( 12   d ), a character  2  “bear” pose image.

TECHNICAL FIELD

The present invention relates to a composite image output apparatus forcombining a picked-up scene image with another and outputting aresulting composite image, and also relates to a composite imagedelivery apparatus for delivering a composite image based upon apicked-up scene image.

BACKGROUND ART

Some types of conventional still cameras or video cameras have thefunction of combining an image picked-up thereby with other images, anddisplaying a resulting composite image.

For instance, such a camera now in practical use outputs a compositeimage formed by combining a picked-up scene image with an animationimage of a flower or a popular character prepared at a fixed or desiredposition in a pickup frame.

An image picked up by a digital camera can be displayed on a displayscreen of a personal computer, combined with a different image andedited. In this case, the different image, for example, can beselectively cut out from another picture file and pasted at a desiredposition in a picked-up image.

However, the problem with such a conventional image processing apparatusis that a prepared image is simply combined at a fixed or desiredposition with a picked-up scene image and then outputted, and thereforethe composite picture excluding the picked-up image tends to be alwaysflat and tiresome.

DISCLOSURE OF INVENTION

The present invention has been made in view of the above-mentionedproblem. It is an object of the present invention to provide an imageoutput apparatus, an image output processing program, and an imageoutput method which makes it possible to not only combine a fixedanimation image with a picked-up scene image to thereby output aresulting composite image, but also combine an animation imagecorresponding to the picked-up scene image with the picked-up sceneimage.

It is another object of the present invention to provide an imagedelivery server, an image delivery program, and an image delivery methodwhich makes it possible to not only deliver a fixed image, but alsodeliver an image.

In order to achieve the above objects, according to one aspect of thepresent invention, there is provided a composite image output apparatuscomprising:

-   -   a picked-up scene image acquiring unit (means) which acquires a        picked-up scene image representing a scene containing at least        one object;    -   a target image extracting unit (means) which extracts a target        image representing any particular one of the at least one object        from the picked-up scene image acquired by the acquiring unit;    -   a corresponding-image obtaining unit (means) which obtains an        image corresponding to the target image extracted by the        extracting unit; and    -   an image outputting unit (means) which adds the image obtained        by the obtaining unit to the picked-up scene image acquired by        the picked-up scene image acquiring unit and outputs a resulting        composite image.

In this composite image output apparatus, when a target imagerepresenting any particular one of the at least one object is extractedby the target image extracting means from the picked-up scene imageacquired by the acquiring means, an image corresponding to the extractedtarget image is obtained by the corresponding-image obtaining means.Thereupon, the image obtained by the obtaining means is added to thepicked-up scene image acquired by the picked-up scene image acquiringmeans to thereby output a resulting composite image. That is, the usercan enjoy creating a delightful composite image.

According to another aspect of the present invention, there is alsoprovided a composite image output apparatus comprising:

-   -   a picked-up scene image acquiring unit (means) which acquires a        plurality of picked-up scene images, each representing a scene        containing at least one object;    -   a target image extracting unit (means) which extracts a target        image representing any particular one of the at least one object        from any selected one of the plurality of picked-up scene images        acquired by said picked-up scene image acquiring unit;    -   a corresponding-image obtaining unit (means) which obtains an        image corresponding to the target image extracted by said        extracting unit; and    -   an image outputting unit (means) which replaces in the picked-up        scene image the extracted target image with the image obtained        by the corresponding-image obtaining unit, for adding the        corresponding-image to a selected one of the others of the        plurality of picked-up scene images from which no target images        are extracted by the target image extracting unit, and for        outputting a resulting composite image.

In this composite image output apparatus, when a target imagerepresenting any particular one of the at least one object is extractedby the target image extracting means from any one of the plurality ofpicked-up scene images acquired by the picked-up scene image acquiringmeans, an image corresponding to the extracted target image is obtainedby the corresponding-image obtaining means. Thereupon, by the imageoutputting means the target image extracted by the target imageextracting means is replaced with the image obtained by thecorresponding-image obtaining means. Also, by the image outputtingmeans, the corresponding-image is then added to a selected one of theothers of the plurality of picked-up scene images from which no targetimages are extracted by the target image extracting means to output aresulting composite image. That is, the user can enjoy creating adelightful image.

According to still another aspect of the present invention, there isalso provided a composite image delivery apparatus comprising:

-   -   a picked-up scene image receiver (receiving means) which        receives from a terminal a picked-up scene image representing a        scene containing an object;    -   a target image extractor (extracting means) which extracts a        target image representing the object from the picked-up scene        image received by the picked-up scene image receiving unit;    -   a corresponding-image obtainer (obtaining means) which obtains        an image corresponding to the target image extracted by the        target image extractor unit;    -   an image adder (adding means) which adds the corresponding-image        obtained by the corresponding-image obtainer to the picked-up        scene image to thereby provide a resulting composite image; and    -   an image deliverer (deliver means) which delivers the composite        image provided by the image adder to the terminal.

In the composite image delivery apparatus, when a picked-up scene imagerepresenting a scene containing at least one object is received from aterminal, a target image representing the object is extracted from thereceived picked-up scene image by the target image extracting means. Animage corresponding to the extracted target image is obtained by thecorresponding-image obtaining means. Thereupon, the obtained image isadded by the image adding means to the picked-up scene image to therebyprovide a resulting composite image, which is then delivered by theimage delivering means to the terminal. That is, the user can enjoycreating a delightful image.

According to a further aspect of the present invention, there is alsoprovided a composite image delivery apparatus that communicates with aterminal, the apparatus extracting a target image contained in apicked-up scene image, sending the terminal pose information on theextracted target image, receiving an image corresponding to the poseinformation that the terminal sent in response to the sending of thepose information by the composite image delivery apparatus, adding thecorresponding image to the picked-up scene image to thereby output aresulting composite image, the apparatus comprising:

-   -   a pose information receiver (receiving means) which receives        from the terminal the pose information for the target image        contained in the picked-up scene image;    -   a corresponding-image storing memory (storing means) having        stored a plurality of images, each corresponding to a respective        one of different items of pose information; and    -   an image deliverer (deliver means) which reads from the        corresponding-image storing memory an image corresponding to the        pose information received by the pose information receiver, and        delivers the read image to the terminal.

In this composite image delivery apparatus, when the terminal extractsthe target image contained in the picked-up scene image and sends poseinformation for the target image to the composite image deliveryapparatus, the composite image delivery apparatus receives the poseinformation in the pose information receiving means thereof. Thereupon,an image corresponding to the received pose information is read by theimage delivering means from the corresponding-image storing means, anddelivered to the terminal. That is, the terminal can receive from theimage delivery apparatus the image corresponding to the target imageextracted from the picked-up scene image and add the image to thepicked-up scene image to thereby output a resulting interestingcomposite image. Thus, the user can enjoy creating a delightful image.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a structure of an electronic circuitof an image output apparatus 10 according to a first embodiment of thepresent invention.

FIG. 2 shows images of “rabbit”, “bear” and “cat” characters takingvarious poses (part 1) stored in a character 1 “rabbit” pose imagememory 12 d, a character 2 “bear” pose image memory 12 e, and acharacter 3 “cat” pose image memory 12 f, respectively, in the imageoutput apparatus 10.

FIG. 3 shows further images of “rabbit”, “bear” and “cat” characterstaking various poses (part 2) stored in the character 1 “rabbit” poseimage memory 12 d, character 2 “bear” pose image memory 12 e, andcharacter 3 “cat” pose image memory 12 f, respectively, in the imageoutput apparatus 10.

FIG. 4 shows still further images of “rabbit”, “bear” and “cat”characters taking various poses (part 3) stored in the character 1“rabbit” pose image memory 12 d, character 2 “bear” pose image memory 12e, and character 3 “cat” pose image memory 12 f, respectively, in theimage output apparatus 10.

FIG. 5 shows images of various poses of a model (part 1) stored in amodel pose memory 12 b in the image output apparatus 10.

FIG. 6 shows images of further various poses of the model (part 2)stored in the model pose memory 12 b in the image output apparatus 10.

FIG. 7 shows images of still further various poses of the model (part 3)stored in the model pose memory 12 b in the image output apparatus 10.

FIG. 8(A) shows, as an example, model images (pose Nos. 1A to 5A)produced based upon a standard body-shaped object image (target image)and stored in a size-adjusted model pose image memory 12 c in the imageoutput apparatus 10.

FIG. 8(B) shows, as another example, model images (pose Nos. 1A to 5A)size-adjusted to a nonstandard body-shaped object image (target image)of FIG. 8(B) and stored in the memory 2C.

FIG. 9 is a diagram showing addition of a “bear” character image as afollower to a picked-up scene image when a follower add mode isspecified by the image output apparatus 10.

FIG. 10 is a diagram showing addition of a “cat” character image in aspace area of a picked-up scene image when a space-area character imageadd mode is specified by the image output apparatus 10.

FIG. 11 is a diagram showing addition of a mirror-reversed “rabbit”character image in a picked-up scene image when a mirror-reversedcharacter image add mode is specified by the image output apparatus 10.

FIG. 12 is a diagram showing replacement of an object image with a“rabbit” character image when a multiple image pickup/replace mode isspecified by the image output, apparatus 10.

FIG. 13 is a diagram showing replacement of an object image with a“rabbit” character image when an object select/replace mode is specifiedby the image output apparatus 10.

FIG. 14 is a flowchart showing an image process (part 1) to be performedby the image output apparatus 10.

FIG. 15 is a flowchart showing an image process (part 2) continued fromthe image process (part 1) of FIG. 14 to be performed by the imageoutput apparatus 10.

FIGS. 16(A) and (B) show flowcharts showing target image extractingprocesses (methods 1 and 2), respectively, included in an image processto be performed in the image output apparatus 10.

FIGS. 17(A) and 17(B) are flowcharts showing corresponding-animationimage acquiring processes (method 1 and 2), respectively, included inthe image process to be performed in the image output apparatus 10.

FIG. 18 is a flowchart showing a process for evaluation of a similarityin pose of a character/model image to an target image of imitationincluded in the corresponding-animation image acquiring process (method1)/(method 2) of FIG. 17(A)/17(B).

FIG. 19 is a diagram showing a specific example of evaluation of asimilarity in pose of a specified character image to a target image in apose similarity estimating process included in thecorresponding-animation image acquiring process (method 1) of FIG.17(A).

FIG. 20 is a diagram showing a specific example of evaluation of asimilarity in pose of a model image to a target image in a posesimilarity estimating process included in the corresponding-animationimage acquiring process (method 2) of FIG. 17(B).

FIG. 21 is a diagram showing another specific example of evaluation of asimilarity in pose (part 2) of a model image to a target image in a posesimilarity estimating process included in the corresponding-animationimage acquiring process (method 2) of FIG. 17(B).

FIG. 22 is a flowchart showing a size adjusting process for a posingmodel by the image output apparatus 110.

FIG. 23 is a block diagram showing a structure of an electronic circuitof an image delivery server 110 in an image delivery system according toa second embodiment of the present invention.

FIG. 24 is a block diagram showing the structure of an electroniccircuit of a terminal 120 in the image delivery system according to thesecond embodiment.

FIG. 25 is a diagram showing replacement of a target image with anotherand addition of a “bear” character image by the image delivery server110 at a request from the terminal device 120 of the image deliverysystem when a multiple-pickup replace mode is specified, according tothe second embodiment.

FIG. 26 is a flowchart showing an image process (part 1) by the terminal120 of the image delivery system according to the second embodiment.

FIG. 27 is a flowchart showing an image process (part 2) continued fromthe image process (part 1) of FIG. 26 by the terminal 120 of the imagedelivery system.

FIG. 28 is a flowchart showing a server process (part 1) by the imagedelivery server 10 of the image delivery system according to the secondembodiment.

FIG. 29 is a flowchart showing a server process (part 2) continued fromthe server process (part 1) of FIG. 28 by the image delivery server 10according to the second embodiment.

FIG. 30 is a flowchart showing a target image extraction processincluded in the server process in the image delivery server 10 accordingto the second embodiment.

FIG. 31 is a flowchart showing a corresponding-animation image acquiringprocess included in the server process in the image delivery server 10according to the second embodiment.

FIG. 32 is a diagram showing evaluation of a similarity in pose of amodel image to a target image, which will be performed in a posesimilarity evaluation process included in the corresponding-animationimage acquiring process of the image delivery system 10 according to thesecond embodiment.

FIG. 33 is a flowchart showing an image process (part 1) by a terminal120 of an image delivery system according to a third embodiment.

FIG. 34 is a flowchart showing an image process (part 2) continued fromthe image process (part 1) of FIG. 33 by the terminal device 120 of theimage delivery system.

FIG. 35 is a flowchart showing a server process by the image deliveryserver 110 of the image delivery system according to the thirdembodiment.

FIG. 36 is a flowchart showing a target image extracting processincluded in the image process in the terminal 120 of the image deliverysystem according to the third embodiment.

FIG. 37 is a flowchart showing a pose number determining processincluded in the image process in the terminal 120 of the image deliverysystem according to the third embodiment.

FIG. 38 is a flowchart showing evaluation of a similarity in pose of amodel image to a target image of imitation in the pose numberdetermining process included in the image process in the terminal 120 ofthe image delivery system according to the third embodiment.

FIG. 39 is shows images of animation “rabbit” characters strikingvarious poses stored in a moving-picture “rabbit” character pose imagememory 12 dM in the image delivery server 110 as posing character imagesto be used for moving pictures in the image delivery server 110according to the third embodiment.

FIG. 40 is a block diagram showing a general structure of the imagedelivery system according to a fourth embodiment.

FIG. 41 is a flowchart showing a user's original character imageregistration process in the image delivery system according to thefourth embodiment, in which FIG. 41(A) is a flowchart showing theprocess to be performed on a portable telephone 200A side, and FIG.41(B) is a flowchart showing the process to be performed on the imagedelivery server 210 side.

BEST MODE FOR CARRYING OUT THE INVENTION

Preferred embodiments of the present invention will be described withreference to the accompanying drawings.

First Embodiment

FIG. 1 is a block diagram showing a structure of an electronic circuitof an image output apparatus 10 according to the first embodiment of thepresent invention.

This image output apparatus 10 includes a computer, which reads aprogram recorded on any of various kinds of recording media or a programreceived externally and controls its own operation in accordance withthe program. The electronic circuit of the computer includes a CPU(central processing unit) 11.

The CPU 11 controls the operation of the components of the circuit inaccordance with an apparatus control program previously stored in memory12, an apparatus control program read into the memory 12 through arecording medium reader 14 from an external recording medium 13 such asa CD-ROM, or an apparatus control program read into the memory 12through an electric transmission controller 15 from a Web server 21 (inthis case, a program server) on the Internet N. The apparatus controlprogram stored in the memory 12 is started up by an input signal issuedin response to a user's operation of an input unit 16 having keys or atouch panel, or by an image signal input through an image input unit 17,or by a communication signal used to communicate with each Web server 21on the Internet N, which is connected through the electric transmissioncontroller 15 to the image output apparatus 10.

Besides the memory 12, the recording medium reader 14, the electrictransmission controller 15, the input unit 16 and the image input unit17, the CPU 11 is connected to a distance sensor 18, a display unit 19including an LED, and an image output unit 20.

The image input unit 17 selectively obtains picked-up scene image datafrom a digital camera 17A, a digital video movie camera 17B, or a VTR(video player) 17C.

The distance sensor 18 detects a distance between the distance sensor 18and an object in a scene whose object image is input to the image inputunit 17.

The image output unit 20 outputs image data processed by this apparatusbased upon the picked-up scene image received through the image inputunit 17. The output image data is selectively output to a memory card20A, a VTR (video recorder) 20B or an image printer 20C to store, recordor print the image.

The memory 12 has stored a system program in charge of control of thewhole operation of the image output apparatus 10 and a communicationprogram to exchange data with each web server 21 on the Internet Nthrough the electric transmission controller 15. The memory 12 also hasstored an image processing program 12 a to perform acombining/outputting process for an image corresponding to a targetimage in the picked-up scene image by the image output apparatus 10through the image input unit 17 and the image output unit 20.

Again, the memory 12 has prepared a model pose image memory 12 b, asize-adjusted model pose image memory 12 c, a character 1 “rabbit” poseimage memory 12 d, a character 2 “bear” pose image memory 12 e and acharacter 3 “cat” pose image memory 12 f. Furthermore, it has prepared apicked-up scene image memory 12 g and an image processing work area 12 hin the work area thereof.

FIG. 2 shows “rabbit”, “bear” and “cat” character images strikingvarious poses (part 1) stored in the character 1 “rabbit” pose imagememory 12 d, the character 2 “bear” pose image memory 12 e, and thecharacter 3 “cat” pose image memory 12 f, respectively, of the imageoutput apparatus 10.

FIG. 3 shows further “rabbit”, “bear” and “cat” character imagesstriking various poses (part 2) stored in the character 1 “rabbit” poseimage memory 12 d, the character 2 “bear” pose image memory 12 e, andthe character 3 “cat” pose image memory 12 f, respectively, of the imageoutput apparatus 10.

FIG. 4 shows still further “rabbit”, “bear” and “cat” character imagesstriking various poses (part 3) stored in the character 1 “rabbit” poseimage memory 12 d, the character 2 “bear” pose image memory 12 e, andthe character 3 “cat” pose image memory 12 f, respectively, of the imageoutput apparatus 10.

Each of the different types of character images striking various posesstored in the character “rabbit”, “bear” and “cat” pose image memories12 d, 12 e and 12 f is used as one imitating a pose of a respective oneof the objects (target) images in a picked-up scene image input from theimage input unit 17. Different pose numbers are stored with differentposes in corresponding relationship.

FIG. 5 shows images of a model corresponding to various model poses(part 1) stored in the model pose memory 12 b in the image outputapparatus 10.

FIG. 6 shows further images of the model corresponding to various modelposes (part 2) stored in the model pose memory 12 b in the image outputapparatus 10.

FIG. 7 shows still further images of the model corresponding to variousmodel poses (part 3) stored in the model pose memory 12 b in the imageoutput apparatus 10.

Each of the model images striking various poses stored in the model poseimage memory 12 b is used to obtain a model image similar in pose to anobject (target) image in a picked-up scene image input from the imageinput unit 17. Different poses are stored with different pose numbers incorresponding relationship and model images of three variations A, B andC are stored for each pose number.

The respective “rabbit”, “bear” and “cat” character images stored in thecharacter 1 “rabbit” pose image memory 12 d, the character 2 “bear” poseimage memory 12 e, and the character 3 “cat” pose image memory 12 fcorresponds to the model images striking the poses indicated by allposes numbers (Nos. 1A, 1B, 1C, 2A, 2B, 2C, . . . nA, nB, nC) stored inthe model pose image memory 12 b.

FIG. 8 shows size-adjusted pose images stored in a size-adjusted modelpose image memory 12 c of the image output apparatus, in which FIG. 8(A)shows, as an example, model pose images (with pose Nos. 1A to 5A)produced to a standard body-shaped object (target) image shown at theleft end of FIG. 8(A) and stored in the memory 12 c. FIG. 8(B) shows, asan example, size-adjusted model pose images (with pose Nos. 1A to 5A)obtained by adjusting the size of the respective standard body-shapedmodel pose images of FIG. 8(A) lengthwise and widthwise to a nonstandardbody-shaped object (target) image and stored in the memory 12 c.

A size-adjusted model image to be stored in the size-adjusted-model poseimage memory 12 c is obtained as follows. A standard body-shaped posingmodel image of a full-length shown by a pose No. 2A in FIG. 8(A) isadjusted in size (lengthwise and widthwise) to a nonstandard body-shapedobject image shown in FIG. 8(B) to become a size-adjusted imagecorresponding to the pose No. 2A. In accordance with a size-adjust ratio(balance) of the size-adjusted model image for the pose No. 2A, allother standard body-shaped model images for the other poses are sizeadjusted to non-standard body-shaped ones, and then stored in thesize-adjusted model pose image memory 12 c.

The picked-up scene image memory 12 g receives picked-up scene imagedata from the image input unit 17 and stores the picked-up scene imagedata in units of a picked-up scene image.

The image processing work area 12 h temporarily stores image data to beprocessed in accordance with the image process program 12 a when theimage data is processed.

Operation of the image output apparatus 10, structured as describedabove, will now be outlined.

FIG. 9 illustrates addition of a “bear” character image as a followerwhen a follower add mode is specified by the image output apparatus 10.

When an object (target) image H is extracted, as shown in FIG. 9(B),from a picked-up scene image 12 g 1 of FIG. 9(A) and a “bear” characterimage is specified as a follower, a “walking” character image “bear” ofpose No. 4 (side view) similar in pose to the object image H as shown inFIG. 9(C) is read from the character 2 “bear” pose image memory 12 e.Thereupon, a position in a picked-up scene image 12 g 1 where the objectimage H was present a certain period of time ago is recognized from thepositions of presence of the object image H on the series of picked-upscene images 12 g 1. The “walking bear” character image (side view) ofthe pose No. 4 read from the character 2 “bear” pose image memory 12 eis added in the picked-up scene image to the position where the objectimage H existed the certain period of time ago, as shown in FIG. 9(D).

That is, the “walking bear” character image similar in pose to theobject image H in the picked-up scene image 12 g 1 is added to theposition in the picked-up image where the “bear” character image followsthe object image H.

FIG. 10 illustrates addition of a “cat” character image in a space areaby the image output apparatus 10 when a space-area character image addmode is specified.

If an object (target) image H1 of imitation is specified and extractedfrom a picked-up scene image 12 g 2 of FIG. 10(A), as shown in FIG.10(B) and a “cat” character image is specified as being added in a spacearea, a “cat” character image of a pose No. 10 (“banzai”) similar inpose to the object (target) image H1 is read from the character 3 “cat”pose image memory 12 f as shown in FIG. 10(C). Then, this read “cat”character image of the pose No. 10 (“banzai”) is added to any randomposition in the space area in the picked-up scene image 12 g 2 wherenone of the objects (target) images H1 and H2 exists, as shown in FIG.10(D).

That is, the “cat” character image imitating the pose of the specifiedobject image in the picked-up scene image 12 g 2 is added to any randomposition in the space area of the picked-up scene image 12 g 2.

FIG. 11 illustrates addition of a mirror-reversed “rabbit” characterimage when a mirror-reversed image add mode is specified by the imageoutput apparatus 10.

If an object (target) image H is extracted, as shown in FIG. 11(B), froma picked-up scene image 12 g 3 including the object (target) image Hconscious of dancing with a partner (not shown), for example, as shownin FIG. 11(A), and a “rabbit” character image is specified as beingadded as a dance partner character image, a “rabbit” character image ofa pose No. 19 (“dance”) similar in pose to the object (target) image His read from the character 1 “rabbit” pose image memory 12 d, as shownin FIG. 11(C). Thereupon, the read “rabbit” character image is mirrorreversed and added as a dance partner to a position in the picked-upscene image 12 g 3 where the mirror-reversed “rabbit” image links withthe object (target) image H for dancing purposes, as shown in FIG.11(D).

That is, the “rabbit” character image imitating the dancing pose of theobject image H in the picked-up scene image 12 g 3 is mirror reversedand added as a dance partner to a position in the picked-up scene imagewhere the mirror-reversed “rabbit” links with object image H.

FIG. 12 illustrates replacement of an object image by a “rabbit”character image when a multiple image pickup/replace mode is specifiedby the image output apparatus 10.

For example, when the user wishes to produce an scene image in which a“rabbit” character as a replacement image gives a kiss to the user, anobject (target) image H1 that is desired to strike a kissing pose isextracted, as shown in FIG. 12(B), from a first picked-up scene image 12g 4 of FIG. 12(A) containing the object image H1 striking the kissingpose. When a “rabbit” character image is specified as a replacingcharacter image, a “rabbit” character image with pose No. 20 (“kiss”)similar in pose to the object (target) image H1, as shown in FIG. 12(C).is read from the character 1 “rabbit” pose image memory 12 d.

Then, as shown in FIG. 12(D), a “rabbit” character image with pose No.20 (“kiss”) similar in pose to the first object image H1 read from thecharacter 1 “rabbit” pose image memory 12 d is added to a secondpicked-up scene image 12 g 4′ containing an object image H2 different inpose from the object (target) image H in the first picked-up image, asshown in FIG. 12(E).

That is, the “rabbit” character image imitating the object image H1 inthe first picked-up scene image 12 g 4 is combined with the object imageH2 in the second picked-up scene image 12 g 4′.

FIG. 13 illustrates replacement of an object image by a character“rabbit” image when an object select/replace mode is selected by theimage output apparatus 10.

If an object image H1 is specified and extracted as one to be replaced,as shown in FIG. 13(B), from a picked-up scene image 12 g 5 containingthree object images H1 to H3, for example, as shown as in FIG. 13(A),and a “rabbit” image is specified as a replacing object image, a“rabbit” character image with pose No. 1 (Say cheese (close-up)) similarin pose to the object image H1 is read from the character 1 “rabbit”pose image memory 12 d, as shown in FIG. 13(C).

On the other hand, the specified object image H1 and other object imagesH2, H3 are cut out from the picked-up scene image 12 g 5, and abackground image 12 g 5′ is produced as illustrated by FIGS. 13(B),13(F), 13(E) and 13(D), respectively.

Thereupon, as shown in FIG. 13(G), the “rabbit” character image similarin pose to the specified object image H1 read from the character 1“rabbit” pose image memory 12 d, and the object images H2, H3 cut out ofthe picked-up scene image 12 g 5 are combined with the background image12 g 5′ at respective proper positions in decreasing order of the imagepickup distances to the respective objects whose images were picked up.

That is, the “rabbit” character image imitating the pose of thespecified object image H1 in the picked-up scene image 12 g 5 replacesthe specified object image H1 to thereby provide a resulting compositeimage.

Then, the image process which will be performed by the image outputapparatus 10 will be described next in more detail.

FIG. 14 is a flowchart showing an image process (part 1) to be performedby the image output apparatus 10.

FIG. 15 is a flowchart showing an image process (part 2) to be performedby the image output apparatus 10, continued from the image process (part1) of FIG. 14.

The type of a character (“rabbit”, “bear” or “cat”) to be used as anadded character or as a replacement in a picked-up scene image 12 g isspecified by manipulation of the input unit 16 (step S1).

An add/compose mode or a replace/compose mode is selected and specifiedfrom among a “1-1: random space-area character image add mode”, a “1-2:follower add mode”, a “1-3: mirror-reversed character image add mode”,and a “2-1: object image select/replace mode”, and a “2-2: multipleimage pickup/replace mode. (step S2).

When a scene image 12 gn picked up by a digital camera 17A, a digitalvideo movie camera 17B or a VTR (player) 17C begins to be input to theimage output apparatus 12 gn (step S3) and a start operation is executedto add the specified character image to the scene image 12 gn (step S4),the process proceeds to extracting the object image in FIG. 16 (stepSA).

FIGS. 16(A) and 16(B) are flowcharts of target image extractingprocesses (methods 1 and 2), respectively, included in the image processin the image output apparatus 10.

In the target image extracting process (method 1) of FIG. 16(A), aseries of picked-up scene images 12 gn, input sequentially atpredetermined timings through the image input unit 17 to the imageoutput apparatus 10, are stored in the picked-up scene image memory 12 gin the work area (step A11).

Thereupon, object images Hn striking different poses corresponding tothe moves of each object are extracted as respective blocks from each ofthe series of picked-up scene images 12 gn stored in the picked-up sceneimage memory 12 g (step A12), and are sorted in the image processingwork area 12 h (step A13).

In the target image extracting process (method 2) in FIG. 16(B), adistance between the distance sensor 18 and each of the objects in ascene whose image is picked up is measured by the distance sensor 18(step A21). The image Hn of each object determined as being present in ashort-distance depending upon the distance between the background andthe object is extracted as a separate block (step A22).

Each extracted object image Hn is then stored as a separate block in theimage processing work area 12 h (step A23).

When each of the object images Hn included in the picked-up scene image12 gn is extracted as a block from the picked-up scene image 12 gn andstored in the image processing work area 12 h (step SA), the picked-upscene image 12 gn with each extracted object image Hn in a broken-linedenclosure M is displayed on the display 19, for example, as shown inFIGS. 9(A), 11(A), 12(A) or 13(A) (step S5).

Then, it is determined whether or not there are a plurality of blocks ofobject image Hn in the picked-up scene image 12 gn displayed on thedisplay 19 (step S6).

If it is determined that there are not a plurality of blocks of objectimage Hn in the picked-up scene image 12 gn, the only one object imageHn extracted from the picked-up scene image 12 gn is set as an objectimage of imitation (step S6→S7 a).

If it is determined that there are a plurality of blocks of object imageHn in the picked-up scene image 12 gn, a target image of imitation isspecified from among the plurality of blocks of object image by theuser's manipulation of the input unit 16 (step S6→S7 b).

When the object image Hn is specified as the target image of imitation,the process proceeds to a corresponding-animation image acquiringprocess in FIG. 17 (step SB).

FIGS. 17(A) and 17(B) are different flowcharts showingcorresponding-animation image acquiring processes (methods 1 and 2),respectively, included in the image process to be performed in the imageoutput apparatus 10.

In the corresponding-animation image acquiring process (method 1) ofFIG. 17(A), a pose similarity of each of the posing images of thespecified character to the object image Hn extracted from the picked-upscene image 12 gn and set as the target image of imitation is evaluated(step B1C), and a posing character image bearing the highest posesimilarity is shown on the display 19 (step B12).

Here, when a posing character image is selected which bears the highestpose similarity to the target image of imitation displayed on thedisplay 19 by the user's manipulation of the input unit 16, thespecified character image (with its pose No.) is registered as ananimation image corresponding to the target image of imitation. As longas there are posing character images with different pose similarities, aposing character bearing a next highest pose similarity is displayedeach time “next display” is clicked (step B13→B14).

On the other hand, if (or each time) the “next display” is clicked bythe user's manipulation of the input unit 16 in a state where thecharacter image bearing the highest pose similarity to the target imageof imitation is displayed on the display 19, a character images bearinga second highest pose similarity will appear (step B13→B15).

When a character image bearing a desired pose similarity is displayedand selected by the user's manipulation of the input unit 16, theselected character image (specified by its pose No.) is registered as ananimation image corresponding to the target image of imitation (stepB13→B14).

That is, in the corresponding-animation image acquiring process (method1), a similarity in pose of the respective posing images of thespecified character to the object image Hn as the target image ofimitation, which has been extracted from the picked-up scene image 12 gnand set as the target image, is directly evaluated. On the basis of thisevaluation, a specified character image is determined, which is used asan added character image or as a replacement striking a pose imitatingthat of the object image Hn.

In the corresponding animation image acquiring process (method 2) inFIG. 17(B), a similarity in pose of each of the posing images of aspecified model stored in the model pose image memory 12 b to the objectimage Hn extracted from the picked-up scene image 12 gn and set as thetarget image of imitation is evaluated (step B2C). A specified characterimage corresponding to the pose No. of a model image bearing the highestpose similarity is shown on the display 19 (step B22).

When a character image is selected which corresponds to a posing modelimage bearing the highest pose similarity to the target image ofimitation displayed on the display 19, by the user's manipulation of theinput unit 16, the character image (specified by its pose No.) isregistered as an animation image corresponding to the target image ofimitation (step B23→B24).

On the other hand, if (or each time) “next display” is clicked by theuser's manipulation of the input unit 16 in a state where a characterimage is displayed on the display 19, which image corresponding to thepose No. of the model image bearing the highest pose similarity to thetarget image of imitation, a specified character image is displayedcorresponding to the pose No. of the model image bearing a secondhighest pose. As long as there are still posing character images withdifferent pose similarities, a posing character bearing a next highestpose similarity is displayed each time “next display” is clicked (stepB23→B25).

Then, when a character image is selected and displayed which correspondsto a model pose image bearing a desired pose similarity and thenselected by the user's manipulation of the input unit 16, the specifiedcharacter image (specified by its pose No.) is registered as ananimation image corresponding to the target image of imitation (stepB23→B24).

As just described above, in the corresponding-animation image acquiringprocess (method 2), a similarity in pose of each of the different posingimages of a basic model of a human figure to the object image Hn as thetarget image of imitation extracted from the picked-up scene image 12 gnand set as such is evaluated. Then, on the basis of this evaluation, aspecified character image is determined, which imitates the pose of theobject image Hn, and is used as an added character or as a replacement.

In this case, compared with the corresponding-animation image acquiringprocess (method 1), in which a similarity in pose of each of the variousposing images of a specified character to the object image Hn as thetarget image of imitation is directly evaluated to thereby determine acharacter image to be used as an added character or as a replacement,the corresponding-animation image acquiring process (method 2), in whicha similarity in pose of the different posing images of the specifiedmodel to the object image Hn extracted from a picked-up scene image 12gn and set as a target image of imitation is evaluated, and a characterimage for use as an added character or as a replacement is determinedfrom a model pose No. selected based upon this evaluation isadvantageous in that the use of a human-figure model makes it easy toobtain a model image bearing a higher pose similarity to the objectimage and obtain a character image bearing a higher pose similarity(imitation).

FIG. 18 is a flowchart showing evaluation of a similarity in pose of acharacter/model image to a target image of imitation in thecorresponding-animation image acquiring processes (methods 1 and 2) ofFIG. 17 included in the image process to be performed in the imageoutput apparatus 10.

When pose similarity evaluation is executed in thecorresponding-animation image acquiring process (method 1), a similarityin pose of each of the various posing images of a specified character tothe target image of imitation is evaluated. When pose similarityevaluation is executed in the corresponding-animation of imitation imageacquiring process (method 2), a similarity in pose of each of thevarious posing images of a specified model to a target image ofimitation is evaluated.

FIG. 19 is a diagram showing an example of evaluation of a similarity inpose of a specified-character image to a target image of imitationincluded in the pose similarity evaluating process of thecorresponding-animation image acquiring process (method 1) to beperformed in the image output apparatus 10.

FIG. 20 is a diagram showing an example of evaluation of a similarity inpose (part 1) of a model image to a target image of imitation includedin the pose similarity evaluate process of the corresponding-animationimage acquiring process (method 2) to be performed in the image outputapparatus 10.

FIG. 21 is a diagram showing an example of evaluation of a similarity inpose (part 2) of a model image to a target image of imitation includedin the pose similarity evaluating process of the corresponding-animationimage acquiring process (method 2).

First of all, in the similarity evaluation process (method 1), aspecified-character image with a first pose No. is read (step C1), forexample, as shown in FIG. 19(A) and the specified-character image andthe target image of imitation H1 are adjusted so as to coincide in size(lengthwise and widthwise), as shown in FIG. 19(X) (step C2).

Thereupon, the adjusted character image and the target image ofimitation H1 are superposed on each other and then a similarity in pose(%) between them is evaluated, and stored as a similarity in pose of thecharacter image to the target image of imitation (step C3).

Here, it is then determined whether the character image (with the firstpose No.) as the target of similarity evaluation is asymmetric (stepC4). If it is determined that the character image is not asymmetric(step C4, “No”), size adjustment and superposition-similarity evaluationbetween the target image of imitation and each of the posing images ofthe specified character having respective successive pose Nos. aresequentially performed, and their similarities are stored, as shown inFIGS. 19(B) and 19(C), for example (step C8→C9→C2-C4).

On the other hand, if it is determined that the character image with thepose No. as the target of similarity evaluation is asymmetric, thecharacter image is mirror reversed (step C4→C5), and the reversedcharacter image and the target image of imitation are adjusted tocoincide in size (lengthwise and widthwise) (step C6).

Thereupon, the target image of imitation and the reversed characterimage are superposed on each other, and a similarity in pose (%) betweenthem is evaluated and stored as the similarity in pose of the reversedimage to the target image of imitation (step C7).

Then, if there are still further other posing images of the specifiedcharacter having successive pose Nos. (step C8), they are sequentiallyread (step C9), and as described above, a similarity evaluation processis performed repeatedly between the respective posing images of thespecified character having the successive pose Nos. and the target imageof imitation (steps C2-C7).

For example, as shown in FIG. 19, when a similarity in pose of the“rabbit” character image to the object image (target image) H1 isevaluated, a “rabbit” character image 12 d{circle over (1)}A with poseNo. 1A is evaluated as bearing the highest pose.

Then, when the similarity evaluation process (method 2) is performed, asshown, for example, in FIG. 20(A) or 21(A), a model image of a firstpose No. {circle over (1)}A is read (step C1), and adjusted in size(lengthwise and widthwise) to the target image of imitation H1 or H, asshown in FIG. 20(X) or FIG. 21(X) (step C2).

Then, the target image of imitation H1 or H and the adjusted model imageare superposed and a similarity in pose (%) between them is evaluatedand stored as the similarity in pose of the model image to the targetimage of imitation (step C3).

Now, it is then determined whether or not the model image as the targetof similarity evaluation is asymmetric (step 4). If it is determinedthat the model posing image 12 b(1A) is not asymmetric, for example, asshown in FIG. 20(A) or 21(A) (step C4, “No”), a model image with a nextpose No. is read (step C8→C9), and in the same manner as describedabove, a pose similarity evaluation process is repeated between thetarget image of imitation H1 or H and the model image of the next pose(steps C2-C4).

On the other hand, as shown, for example, in FIG. 20(B) or 21(B) if itis determined that a model image 12 b(1B) or 12 b(4B) with pose No. 1Bor 4B as the target of similarity evaluation in this instance isasymmetric, the model image with pose No.{circle over (1)}B or {circleover (4)}B is mirror reversed (step C4 to C5), and as shown in FIG.20(C) or 21(C), this reversed model image 12 b(1B)R or 12 b(4B)R isadjusted in size (lengthwise and widthwise) to the target image ofimitation H1 or H (step C6).

Then, the target image of imitation H1 or H and the reversed andsize-adjusted model image 12 b(1B)R or 12 b(4B)R are superposed on eachother, and a similarity in pose (%) between them is evaluated stored asthe similarity in pose of the posing reversed model image 12 b(1B)R or12 b(4B)R to the target image of imitation (step C7).

Then, if there are further model images with successive pose Nos. (stepC8), these model images are sequentially read (step C9), and in the samemanner as described, the pose similarity evaluation process is repeatedbetween the target image of imitation and the respective model images(steps C2-C7).

Consequently, as shown, for example, in FIG. 20, if a pose similarityevaluation is performed between the object image (target image) H1 andthe respective posing model images, a model image 12 b(1B) with pose No.1B is evaluated as one bearing the highest pose similarity. As shown inFIG. 21 for example, if pose similarity evaluation is carried outbetween the object image (target image) H and the respective modelimages, a model image 12 b(4B)R with pose No. {circle over (4)}BR isevaluated as one bearing the highest pose similarity.

As described, when a specified character image imitating that of theobject image Hn as the target image of imitation extracted from thepicked-up scene image 12 gn and set as such is determined in thecorresponding-animation image acquiring process (step SB) as describedwith reference to FIGS. 17-21, it is then determined whether a composemode for the specified image is an add/compose mode or otherwise areplace/compose mode (step S8).

If it is determined that the compose mode for the specified image is the“add/compose mode” and the “1-1: random space-area character image addmode” (step S8 to S9, “1-1”), a “cat” character image 12 f{circle over(10)} which imitates a pose “banzai” of the object image H1 as thetarget of imitation obtained in the corresponding-animation imageacquiring process (see FIG. 17) is added to and displayed at a randomposition in a space area other than in an area of the extracted objectblocks M in the picked-up scene image 12 g 2′, as shown in FIG. 10(D),for example (step S11 a).

At this time, if the manual operation is performed to move the “cat”character image 12 f{circle over (10)} from its present position wherethe “cat” character image is added in the picked-up scene image 12 g 2′to any specified different position (step S16 to S17), the “cat”character image is then added again to the different position and aresulting composite image is displayed (step S18).

Thereupon, data on the composite image is output from the image outputunit 20 (step S19), recorded on a memory card 20A or a VTR (recorder)20B or otherwise printed by an image printer 20C on demand.

If it is determined that the specified image compose mode is the“add/compose mode” and the “1-2: follower add mode” (step S8 S9, “1-2”),as shown, for example, in FIG. 9 a reversed “bear” character image 12e{circle over (4)}R imitating a “walking” pose of the object image(target image) H in the picked-up scene image 12 g 1, obtained in thecorresponding-animation image acquiring process (see FIG. 17), is addedto and displayed at a position in the picked-up scene image 12 g 1′where the “bear” character image, so to speak, follows the object(target image) H, in other words, at the position of the object image(target image) H in a relevant one of the series of picked-up sceneimages 12 g 1 a certain time ago, as shown in FIG. 9(D) (step S11 b).

At this time, if the added and displayed “bear” character image 12e{circle over (4)}R is moved by a manual operation from its presentposition to any specified different position (step S16→S17), the “bear”character image is moved to the different position to thereby display aresulting composite image (step S18).

Thereupon, image data on this composite image is output from the imageoutput unit 20 (step S19), and recorded on a memory card 20A or the VTR(recorder) 20B or otherwise printed out by the image printer 20C ondemand.

If it is determined that the compose mode for the specified image is the“add/compose mode” and the “1-3: mirror-reversed character image addmode (step S8 to S9 “1-3”), as shown, for example, in FIG. 11 the“rabbit” character image 12 d{circle over (19)}A (FIG. 11(C)) imitatinga dancing pose of the object (target) image H in the picked-up sceneimage 12 g 3, obtained in the corresponding-animation image acquiringprocess (see FIG. 17), is mirror reversed, added to a position where thereversed “rabbit” character image links with the object (target) image Hstriking a dancing pose in the picked-up scene image 12 g 3′, and aresulting composite image is displayed, as shown in FIG. 11(D) (step S11c).

When in this case a process for moving the mirror reversed “rabbit”character image 12 d{circle over (19)}R displayed at the position in thepicked-up scene image 12 g 3 in which the mirror reversed “rabbit”character image links with the object (target) image H from its presentposition to any specified different position is performed manually (stepS16→S17), the mirror-reversed “rabbit” image 12 d{circle over (19)}R ismoved to the different position to thereby provide a resulting compositeimage, which is then displayed (step S18).

Thereupon, data on this composite image is outputted from the imageoutput unit 20 (step S19), recorded on the memory card 20A or VTR(recorder) 20B or otherwise printed out by the image printer 20C ondemand.

If it is determined that the specified image compose mode includes the“replace/compose mode” and the “2-1: object select/replace mode” (stepSB8→S10, “2-1”), the specified object (target) image H1 as the targetimage of imitation specified in the picked-up scene image 12 g 5acquired in the corresponding-animation image acquiring process (FIG.17) is replaced with a “rabbit” character image 12 d{circle over (1)}imitating the pose of the object image H1 (step SB18), and a resultingcomposite image is then displayed, as shown in FIG. 13(G).

In this case, three object images, that is, the object (target) image H1and other two object images H2 and H3 are cut out from the picked-upscene image 12 g 5 to thereby provide a picked-up background image 12 g5′. The “rabbit” character image 12 d{circle over (1)} similar in poseto the object image H1, the remaining two object images H2 and H3 cutout from the scene image 12 g 5 are then added to the respective properpositions in the picked-up background image 12 g 5′ in decreasing orderof the image pickup distances to the respective objects concerned sensedby the distance sensor 18 in the image pickup to thereby provide aresulting composite image.

Without producing the remaining background image 12 g 5′, the “rabbit”character image 12 d{circle over (1)} similar in pose to the object(target) image H1 and the other two images H2 and H3 may be added to thetheir proper positions in the picked-up scene image 12 g 5 in decreasingorder of the image pickup distances to the respective objects concernedsensed by the distance sensor 18 in the image pickup to thereby providea resulting composite image.

When the process for moving the “rabbit” character image 12 d{circleover (1)} added in place of the target image H1 in the picked-up sceneimage 12 g 5″ from its present position to any specified differentposition is performed (step S16→S17), the “rabbit” character image 12d{circle over (1)} is moved to the specified different position in thepicked-up scene image 12 g 5″ to thereby provide a resulting compositeimage, which is then displayed (step S18).

Thereupon, data on the composite image is outputted from the imageoutput unit 20 (step S19), recorded on the memory card 20A or VTR(recorder) 20B or otherwise printed by the image printer 20C, on demand.

If it is determined that the image compose mode includes the“replace/compose” mode and the “2-2: multiple image pickup/replace mode”(step S8→S10, “2-2”), the position of the object (target) image H1 inthe first picked-up scene image 12 g 4 is stored, for example, as theone where the object image H1 is to be replaced with an imitating image,as shown in FIG. 12 (step S10→S13).

When a second scene image 12 g 4′ is picked up (step S14), a “rabbit”character image 12 d(20) imitating the pose of the object image H1acquired in the corresponding-animation image acquiring process (FIG.17) is added in the second-time picked-up scene image 12 g 4′ to aposition corresponding to the stored position of the object (target)image H1 in the first picked-up scene image 12 g 4 such that the“rabbit” character image 12 d(20) links with the object image H1 and aresulting composite image 12 g 4″ is then displayed (step S15).

When the process for moving the “rabbit” character image 12 d(20) fromits present position to any specified different position is performedmanually in the second picked-up scene image 12 dg 4′ (step S16→S17),the “rabbit” character 12 d(20) is moved to the specified differentposition and a resulting composite image is displayed (step S18).

Thereupon, image data on this composite image is then outputted from theimage output unit 20 (step S19), recorded on the memory card 20A or thevideo tape recorder 20B or otherwise printed out by the image printer20C on demand.

As will be obvious from the above, according to the image processingfunction of the image output apparatus 10, when an object image isextracted from the scene image received through the image input unit 17,any particular type of character image similar in pose to the extractedobject image is selected from among the character 1 “rabbit” pose imagememory 12 d, the character 2 “bear” pose image memory 12 e and thecharacter 3 “cat” pose image memory 12 f, the selected particular typeof character image replaces the object image in the picked-up sceneimage, or is added to the picked-up scene image to thereby provide aresulting composite image. That is, a very delightful scene image whichuses the animation image imitating the pose of the target object as anadded one or otherwise as one replacing the target object is easilycreated without creating a scene image to which a fixed animation imageis simply added in the image pickup frame.

A function of the image output apparatus 10 that adjusts the size of amodel image will be described next. In the function of adjusting thesize of the model image, the posing model images coincident in size tothe standard body-shaped object (target) images and stored in the modelpose image memory 12 b are adjusted in size to a non-standardbody-shaped object (target) image and stored as size-adjusted modelimages in the size-adjusted model pose image memory 12 c. In thecorresponding-animation image acquiring process (part 2), asize-adjusted model image stored in the adjusted-model pose image memory12 c is used to evaluate a similarity in pose of the adjusted modelimage to an object image as the target image of imitation to therebyobtain a corresponding specified-character image bearing that posesimilarity. This allows a character image similar in pose to an objectimage having any body shape to be obtained easily.

FIG. 22 is a flowchart of a posing model size adjust process to beperformed by the image output apparatus 10.

When a picked-up image representing a scene containing an objectincluding, for example, a non-standard body-shaped full length figure ofthe user shown in FIG. 8(B) is inputted via the image input unit 17(step D1), a image of the user as an object is extracted in the targetimage extracting process (method 1 or 2) of FIG. 16 (step DA) andcompared to a model image 12 b{circle over (2)}A of a standardbody-shaped full length stored in the model pose image memory 12 b (stepD2).

Then, the model image 12 b{circle over (2)}A is adjusted in size to theobject image extracted from the picked-up scene image with respect tothe longitudinal proportions of face, body and legs (step D3) and thenwith respect to the width proportions of face, body and legs (step D4).

Thereupon, all other posing model images stored in the model pose imagememory 12 b are adjusted in size to the size-adjusted model image 12b{circle over (2)}A and then registered in the size-adjusted model poseimage memory 12 c (step D5).

As described above, a similarity in pose of any one of the size-adjustedmodel images stored in the size-adjusted model pose image memory 12 c toan object image as the target image of imitation is evaluated and acharacter image bearing a similarity equal in pose to the object imageis obtained such that character images similar in pose to object imageshaving all kinds of body shapes can be easily obtained. A characterimage more similar in pose to the object image as the target object ofimitation can be added to the picked-up scene image or replace theobject image in the picked-up scene image to thereby provide a resultingcomposite image.

While in the embodiment a character image similar in pose to the objectimage extracted from the picked-up scene image is selectively obtainedand its animation image is used as an added one or a replacement in thepicked-up scene image to thereby output a resulting composite image,face images having various expressions and persons' images actuallypicked-up may be stored beforehand and used individually as an added oneor a replacement imitating the pose of the object image extracted fromthe picked-up scene image in the picked-up scene image to thereby outputa resulting composite image.

The present invention can be implemented well in a digital camera. Ifthe present invention is implemented, especially in a digital camerawith a liquid crystal display, an object image can be picked up so thata resulting composite image may be confirmed on the spot.

While in the embodiment the picked-up scene image as a still image hasbeen illustrated, object images changing in time series relative to thebackground image of dynamic images inputted via the image input unit 17,for example, a digital video movie 17B may be sequentially extractedfrom the dynamic images. Character images similar in pose to thesequentially extracted object images, respectively, may be selectivelyacquired and added to the picked-up dynamic images or replace the objectimages in the respective dynamic images to thereby provide a resultingseries of composite dynamic images.

The image processing function of the embodiment may be used to replace acharacter image as a target image of imitation specified and extractedfrom animation images, for example, used in a game with a prepareddesired character image or a person's image picked up actually tothereby display a resulting composite image.

The respective processes performed by the image output apparatus 10described in the embodiment, that is, any one of the image process shownin the flowcharts of FIGS. 14 and 15, the object image extractingprocess included in the image process shown in the flowchart of FIG. 16,the corresponding-animation image acquiring process included in theimage process shown in the flowchart of FIG. 17, the process forevaluating the similarity in pose of a character (or model) image to theobject image shown in the flowchart of FIG. 18 included in thecorresponding-animation image acquiring process, and the posing modelsize-adjusting process shown in the flowchart of FIG. 22 can be storedand distributed as programs, which a computer can execute, on memorycards (ROM and RAM cards, etc.), magnetic discs (floppy discs, harddiscs, etc.), optical discs (CD-ROMs, DVDs, etc.), or external recordingmediums 13 such as semiconductor memories, etc. Various computerterminals having a communication function with a communication network(the Internet) N can read the programs stored in the external recordingmediums 13 through the recording medium reader 14 into the memories 12and control its operation in accordance with the read programs tothereby perform the image processing functions described in theembodiment and perform processes similar to the methods mentioned above.

Data included in the programs to perform the respective above-mentionedprocesses can be transmitted via the communication network (theInternet) N in the form of a program code. The program data can be takenfrom the computer terminals connected to the communication network (theInternet) N to thereby perform the above-mentioned image processingfunctions.

Second Embodiment

FIG. 23 is a block diagram indicative of the composition of anelectronic circuit of an image delivery server 110 in an image deliverysystem of the invention.

FIG. 24 is a block diagram indicative of the composition of anelectronic circuit of a terminal 120 in the image delivery systemaccording to the second embodiment.

The image delivery system provides image creating/delivering servicesfrom the image delivery server 110 to various terminals 120 such as aportable telephone 120A, a personal digital assistant (PDA) 120B, apersonal computer (PC) 120C, a digital camera 120D, etc., connected to acommunication network such as the Internet or Bluetooth (R). The imagedelivery server 110 delivers to each of the terminals 120 an imagecorresponding to a target image in an image picked up by the terminal120 (120A-120D) (for example, a character image corresponding to aperson image in a picked-up scene image).

In FIG. 23, the image delivery server 110 includes a computer that readsprograms recorded on various recording mediums or programs receivedexternally with the computer operation being controlled by the readprograms. The computer includes a CPU (Central Processing Unit) 111 inits electronic circuit.

The CUP 111 controls the respective elements of the circuit inaccordance with a server control program beforehand stored in a memory112, a server control program read into the memory 112 via a recordingmedium reader 114 from an external recording medium 113 such as a CD-ROMor a server control program read into the memory 112 via an electrictransmission controller 115 from another Web server (in this case, aprogram server) on the Internet N. The server control program stored inthe memory 112 is started up by a communication signal from eachterminal 120 (120A-120C) on the Internet N connected via thetransmission controller 115, a communication signal from the digitalcamera 120D connected via a short-distance wireless communication unit116 or an input signal produced by an operator from an input unit 117including keys or a touch panel.

The CPU 111 is connected to the memory 112, recording medium reader 114,transmission controller 115, short-distance wireless communication unit116, and input unit 117 as well as a display unit 118 including an LCDor a CRT.

In addition to the system program in charge of the entire operation ofthe image delivery server 110, a communication program for datatransmission with the respective terminals 120 (120A-120C) on theInternet. N via the transmission controller 115, and a short-distancecommunication program for data communication with digital cameras 120Din a communication range of the short-distance wireless communicationunit 16 via the communication unit 16, the memory 112 has stored animage delivery program 112 a to produce an image corresponding to anobject image in the picked-up image obtained from a respective one ofthe terminals 120 (120A-120D) communication connected via thetransmission controller 115 and the short-distance wirelesscommunication unit 116, add the corresponding image to the picked-upimage and deliver a resulting image to the terminal.

The memory 112 has prepared a model pose image memory 112 b, asize-adjusted model pose image memory 112 c, a character 1 “rabbit” poseimage memory 112 d, a character 2 “bear” pose image memory 112 e, acharacter 3 “cat” pose image memory 112 f, and a requester-terminalinformation memory 112 g as well as in its work area a picked-up sceneimage memory 112 h and an image processing work area 112 i.

The character “rabbit”, “bear” and “cat” images striking respectiveposes stored in the characters “rabbit”, “bear” and “cat” pose imagememories 112 d, 112 e and 112 f, respectively, of the image deliveryserver 110 are identical to those illustrated in FIGS. 2-4 of the firstembodiment, and FIGS. 2-4 are also used to describe the secondembodiment.

Each of the various posing images of the characters stored in thememories 112 d, 112 e and 112 f is used as an imitating image similar inpose to an object (target) image in a picked-up scene image inputted ata request of a respective one of the external terminals 120 (120A-120D)communication connected to the image delivery server 110. Each posingcharacter image is stored with its corresponding pose No.

FIGS. 5-7 relate to the first embodiment and are also used to illustratethe second embodiment here. FIGS. 5-7 show model images striking therespective poses stored in the model pose image memory 112 b of theimage delivery server 110.

Each of the various posing model images stored in the memory 112 b isused to obtain a model image similar in pose to an object (target) imagein a picked-up scene image inputted at a request of a respective one ofthe external terminals 120 (120A-120D) communication connected to imagedelivery server 110 with the different poses being stored withcorresponding pose numbers. In addition, three different variations ofmodel images A, B and C are stored for each pose number.

The character images stored in each of the character 1 “rabbit” poseimage memory 112 d, the character 2 “bear” pose image memory 112 e, andthe character 3 “cat” pose image memory 112 f are the ones strikingposes corresponding to all the pose Nos. (Nos. 1A, 1B, 1C, 2A, 2B, 2C, .. . , na, nb, nc) stored in the model pose image memory 112 b.

A size-adjusted model image of pose No. 2A to be stored in asize-adjusted model pose image memory 112 c is obtained by verticallyand horizontally adjusting the size of a standard body-shaped modelimage of a full length figure striking a pose shown by a pose No. 2A inFIG. 4 to a nonstandard body-shaped object (target) image. All otherstandard body-shaped model images striking respective poses are sizeadjusted in accordance with a vertical-horizontal ratio of thesize-adjusted model image with pose No. 2A and stored in thesize-adjusted model pose image memory 112 c.

A size-adjusted model image to be stored in the size-adjusted-model poseimage memory 112 c is obtained as follows. A standard body-shaped posingmodel image of a full-length shown by pose No. 2A in FIG. 8(A) isadjusted in size (lengthwise and widthwise) to a nonstandard body-shapedobject image shown in FIG. 8(B) to become a size-adjusted imagecorresponding to pose No. 2A. In accordance with a size-adjust ratio(balance) of the size-adjusted model image for pose No. 2A, all otherstandard body-shaped model images for the other poses are size adjustedto the non-standard body-shaped ones, and then stored in thesize-adjusted-model pose image memory 12 c.

A requesters terminal information memory 112 g has registered IDs, bankaccount numbers, and passwords of the terminals 120 (120A-120D), astheir individual mail-order business information about image deliverymembers who have requested the image delivery server 110 to registerthem as such.

The picked-up scene image memory 112 h temporarily stores, in units of apicked-up scene image, picked-up scene image data inputted from therespective external terminals 120 (120A-120D) communication connected tothe image delivery server 110.

The image processing work area 112 i temporarily stores image dataprocessed in accordance with the image delivery program 112 a dependingupon the image processing.

In FIG. 24, each terminal 120 includes a computer that reads a programrecorded on each type of recording medium or an externally receivedprogram with the computer operation being controlled by the program. Itselectronic circuit includes a CPU (Central Processing Unit) 121.

CUP 121 controls operation of the respective circuit elements inaccordance with a terminal control program read beforehand into thememory 122, a terminal control program read into the memory 122 via arecording medium reader 124 from an external recording medium 123 suchas a CD-ROM or a ROM card, or a terminal control program read into thememory 122 via an electric transmission controller 125 from a Web server(in this case, program server) on the Internet N. The terminal controlprogram stored in the memory 122 is started up by an input signalproduced by a user's operation on an input unit 126 including keys or atouch panel, an image signal inputted by a digital camera 132 through animage input unit 127 a, a communication signal with each Web server suchas an image delivery server 110 on the Internet N connected via thetransmission controller 125 to the terminal 120, a communication signalwith the image delivery server 110 connected via a short-distancewireless communication unit 128 to the terminal 120, or apresent-position detection signal indicative of the present position ofthe moving terminal 120 from a GPS (Global Positioning System) 129.

The CPU 121 is connected to the memory 122, recording medium reader 124,transmission controller 125, input unit 126, image input unit 127 a,short-distance wireless communication unit 128, and GPS 129 as well as adisplay unit 130 of LCD and an image printer 131 through an image outputunit 127 b.

The image input unit 127 a receives picked-up scene image data from anexternally connected or built-in digital camera 132. For example, if theterminal 120 includes a portable telephone 120A, a personal digitalassistant (PDA) 120B or a digital camera 120D, the image input unit 127receives the picked-up scene image data from the built-in digital camera132. If the terminal 120 includes a personal computer (PC) 120C, theimage input unit 127 receives the picked-up scene image data from theexternally connected digital camera 132.

The image output unit 127 b outputs image data processed and deliveredby the image delivery server 110 based upon the picked-up scene 25 imageinputted by the image input unit 127 a at a request of the terminal. Theoutputted image data is then delivered to and printed by the imageprinter 131.

In addition to the system program in charge of control of the wholeoperation of the terminal 120 (120A-120D), a communication program fordata communication with each Web server such as the image deliveryserver 110 on the Internet N via the transmission controller 125, and ashort-distance communication program for data communication with theimage delivery server 110 present in a communication range of theshort-distance wireless communication unit 128 via the wirelesscommunication unit 128, the memory 122 has stored a communication/imageprocessing program 122 a for requesting the image delivery server 110 toperform image processing based upon the picked-up scene image inputtedfrom the image input unit 127 a, for displaying image data delivered bythe image delivery server 110, and for performing a printing processthrough the image output unit 127 b.

The terminal 120 (120A-120D) includes in the memory 122 a model poseimage memory 122 b and a size-adjusted-model pose image memory 122 csimilar in specification to those memories 112 b and 112 c included inthe memory 112 of the image delivery server 110 of the secondembodiment.

The memory 122 includes an image processing work area 122 d thattemporarily stores image data processed in accordance with thecommunication/image processing program 122 a depending upon itsprocessing.

Operation of the image delivery system of the second embodiment will beoutlined next.

FIG. 11 relates to the first embodiment and is also applied to describethe second embodiment. FIG. 11 illustrates addition of a mirror-reversedcharacter image “rabbit” in a picked-up scene image when the terminal120 has specified a mirror-reversed character image add mode andrequested for image delivery from the image delivery server 110.

As shown in FIG. 11(A), the terminal 120 picks up a scene image 12 g 3containing an object image (target image) H striking a pose, forexample, beforehand conscious of dancing with a partner, and requeststhe image delivery server 110 to create a composite image in which amirror-reversed character image dances with the target image byspecifying a mirror-reversed character image add mode, and deliver it tothe terminal 120.

Then, as shown in FIG. 11(B), in the image delivery server 110 theobject (target) image H is extracted from the picked-up scene image 12 g3 sent by the terminal 120. When a “rabbit” character image is specifiedas, an added one, a “rabbit” character image with pose No. 19 (“dance”)similar in pose to the object (target) image H is read out from thecharacter 1 “rabbit” pose image memory 112 d, as shown in FIG. 11(C).Thereupon, this “rabbit” character image is mirror reversed and added inthe picked-up scene image 12 g 3′ to a position symmetrical to theposition of the object (target) image in respect to a vertical centerline, and where the mirror-reversed “rabbit” image links with the object(target) H for dancing purposes, as shown in FIG. 11(D). This produces acomposite image which is then delivered to the requester's terminal 120and displayed or printed out there.

That is, the image delivery server 110 mirror reverses the “rabbit”character image imitating the dancing pose of the object (target) imageH in the scene image 12 h 1 picked up by the terminal 120, and adds themirror-reversed “rabbit” character image as a dance partner to aposition in the picked-up scene image 12 h 1 where the mirror-reversed“rabbit” character image links with the object (target) image strikingthe dancing pose for dancing purposes. A resulting composite image isthen sent to the terminal 120.

FIG. 25 illustrates replacement of an object image with a “bear”character image, which will be performed by the image delivery server110 when the terminal 120 of the image delivery system has requestedimage creation and delivery from the server and specified a multipleimage pickup/replace mode.

When the user at the terminal 120 desires to create a scene image inwhich the object image and a “bear” character image wave good-by to eachother, the user picks up a scene image containing the object (target)image H1 striking the same “good-by” pose as the character image isdesired to strike, as shown in FIG. 25(A), sends the picked-up sceneimage to the image delivery server 110, and requests the image deliveryserver 10 to create a composite scene image based upon the picked upscene image 12 h 2 by specifying the multiple image pickup/compose modeand deliver the composite scene image to the terminal.

As shown in FIG. 25(B), the image delivery server 110 extracts the image(target image) H1 from the first picked-up image 12 h 2 sent by theuser. When a “bear” character image is specified as being a replacingone by the user, a “bear” character image (striking a pose No. 11“Good-by”) similar in pose to the object (target) image H1 is read outfrom the character 2 “bear” pose image memory 112 e. In this case, whenthe present time is sensed, for example, as “08:00 p.m.” or it isdetermined that it is now at night, the “bear” character image strikingthe pose “Good-by” is read out, for example, as a night character image12 e{circle over (11)}N sitting on the moon.

Then, as shown in FIG. 25(D), when a second scene image 12 h 2′containing a target image H2 as a partner corresponding to the objectimage (target image) H1 in the first picked-up scene image 12 h 2 andstriking another pose is picked up by the terminal 120, the read “bear”character image striking the pose No. 11 (“Good-by”) for use at night isadded to the second picked-up scene image 12 h 2′, to thereby provide aresulting composite image shown as 12 h 2″ in FIG. 25(E). The resultingcomposite image is then delivered to the requester's terminal 120 anddisplayed or printed out there.

That is, in the image delivery server 110 the character image imitatingthe good-by pose of the object (target) image H1 in the first sceneimage 12 h 2 picked up by the terminal 120 is added to a proper positionrelative to the position of the object (target) image H2 in a secondscene image 12 h 2′ picked up by the terminal 120 to thereby provide aresultant composite image, which is then delivered to the terminal 120.

While FIG. 13 was used for explanation of the object replacing processof the image output apparatus 110 of the first embodiment, it is alsoapplied for explanation of a similar object replacing process of theimage delivery server of the second embodiment.

FIG. 13 illustrates replacement of an object image with a “rabbit”character image by the image delivery server 100 when the terminal 120requested image creation and delivery from the image delivery server andspecified an object image select/replace mode.

As shown in FIG. 13(A), the terminal 120 picks up a scene image 12 g 5containing, for example, three person (object) image H1-H3, sends thescene image along with notice of a specified replacing character image,for example, of “rabbit” to the image delivery server 110, and requeststhe image delivery server 110 to create a composite image based upon thepicked-up scene image by specifying an object image select/replace mode,and then to deliver it to the terminal 120.

Thereupon, as shown in FIG. 13(B), the image delivery server 110extracts from the picked-up scene image 12 g 5 sent by the terminal 120an object (or target) image H1 specified as being replaced by theterminal 120. When a replacing “rabbit” character image is specified, a“rabbit” character image similar in pose (pose No. 1 “Say cheese”(close-up)) to the object (target) image H1 is read out from thecharacter 1 “rabbit” pose image memory 112 d, as shown in FIG. 13(C).

As shown in FIGS. 13(B), (D), (E) and (F), the specified object (target)image H1 and other object images H2 and H3 are cut out from the sceneimage 12 g 5 and the background image 12 g 5′ remains.

Thereupon, as shown in FIG. 13(G), the “rabbit” character image similarin pose to the specified object image (target image) H1 read out fromthe character 1 “rabbit” pose image memory 112 d and the other twoobject images H2 and H3 cut out from the scene image 12 g 5 are added toproper positions in the background image 12 g 5′ in decreasing order ofthe image pickup distances to the respective objects concerned tothereby provide a resulting composite image 12 g 5″. The composite image12 g 5″ is then delivered to the requester's terminal 120 and displayedor printed out there.

That is, in the image delivery server 110 the “rabbit” character imageimitating the pose of the specified object (target) image H1 in thescene image 12 g 5 picked up by the terminal 120 replaces the object(target) image H1 in the scene image 12 g 5 picked up by the terminal120, and a resulting composite image is delivered to the terminal 120.

In the image delivery system, the image process modes include a followeradd mode and a space-area character image add mode whose illustrativedescriptions using the drawings are omitted in the outline of theoperation of the image delivery server, in addition to thereversed-character image add mode, multiple image pickup/replace modeand object image select/replace mode.

In the follower add mode, when the terminal 120 picks up a scene image12 hn containing an object (target) image H walking or striking awalking pose and requests the image delivery server 100 to create acomposite scene image based upon the scene image 12 hn by specifying thefollower add mode and a character image as the follower imitating thepose of the object image H and to deliver the composite scene image tothe terminal 120, the image delivery server 110 adds the specifiedcharacter image in the scene image 12 hn to a position where thespecified character image as the follower follows the object (target)image H to thereby deliver a resulting composite image to the terminal120.

When the terminal 120 picks up a scene image 12 hn containing any object(target) image H, sends this scene image 12 hn and a specified characterimage imitating the pose of the specified object (target) image H to theimage delivery server 110, and requests the delivery server 110 tocreate a composite scene image based upon the scene image and thespecified character image H in the picked-up scene image 12 hn byspecifying the space-area character image add mode and to deliver thecomposite image to the terminal, the image delivery server 110 adds thespecified character image to a random position in a space area of thescene image 12 hn to thereby deliver a resulting composite image to theterminal 120.

The details of the image processing and delivering operation by theimage delivery server 110 and the terminal 20 will be described next.

FIG. 26 is a flowchart of an image process (part 1) to be performed bythe terminal 120.

FIG. 27 is a flowchart of an image process (part 2) continued from theimage process (part 1) of FIG. 26. FIG. 28 is a flowchart of a serverprocess (part 1) to be performed by the image delivery server 110.

FIG. 29 is a flowchart of a server process (part 2) continued from theserver process (part 1) of FIG. 28.

First, the terminal 120 (120A, 120B or 120C) responds to themanipulation of its input unit 126 to connect via the transmissioncontroller 125 or short-distance wireless communication unit 128 to theimage delivery server 110 on the Internet N or in a short-distancewireless communication range and to send the user's ID and passwordthereof to the image delivery server 110 (step TA1).

When the image delivery server 110 receives the ID and password from theterminal 120 (120A, 120B or 120C) via the transmission unit 115 throughthe Internet N or from the terminal (digital camera) 120D via theshort-distance wireless communication unit 116 (step SB1), it determineswhether or not the terminal is a full member depending on whether theterminal ID and password can find their matches among the members' onesregistered in the requester terminal information memory 112 g of thememory 112 (step SB2).

When it determines that the ID and password of the terminal 120 can notfind their matches among the registered members' IDs and passwords andthat the terminal is not a full member, the connection of the terminal120 to the image delivery server 110 is interrupted and an error processis performed (step SB2, “No.”).

When the connection of the terminal 120 to the image delivery server 110is not interrupted over a predetermined time after the terminal 120 hassent its ID and password to the image delivery server 110 (step TA1),data representing the present position of the terminal 120 (in the caseof the portable telephone 120A, data representing the position of itsbase station concerned) sensed by GPS 129 is sent from the terminal 120to the image delivery server 110 (step TA2).

When the image delivery server 110 receives from the terminal 120 datarepresenting the present position of the terminal (step SB3), itdetermines whether the terminal 120 is at present in a predeterminedparticular place (for example, a souvenir picture pickup place in atheme park or tourist spot) or whether the present time is in aspecified time zone (for example, at night) (step SB4).

If the image delivery server 110 determines that the terminal 120 is atpresent in the predetermined particular place or that the present timeis in the specified time zone (step SB4), a particular charactercorresponding to the particular place (for example, in the case of atheme park, a popular character there) or a particular charactercorresponding to the specified time zone (for example, in the case of atnight, a “bear” character sitting on a horned moon) is determined as thespecified character corresponding to the target object and the name ofthis character is then sent to the terminal 120 (step SB5).

When the image delivery server 110 determines that the terminal 120under connection is not in the predetermined place, or that the presenttime is not in the specified time zone (step SB4), the image deliveryserver 110 waits for reception of the type of a specified character fromthe terminal 120 (step SB6).

If the terminal 120 receives the name of the particular characterdetermined and sent by the image delivery server 110 because theterminal 120 is in the particular place or the present time is in thespecified time zone (step TA3), the particular character name isdelivered to and displayed by the display unit 130 (step TA4).

When the terminal 120 does not receive the name of the particularcharacter from the image delivery server 110 because the terminal 120 isnot in the particular place or the present time is not in the specifiedtime zone (step TA3), the user of the terminal 120 manipulates the inputunit 126 to thereby specify the type of a character (“rabbit”, “bear” or“cat”) image to be used as an added one, or a replacement, in thepicked-up scene image and sends the specified type of a character imageto the image delivery server 110 (step TA5).

Then, the terminal 120 selects the add or replace/compose mode fromamong the “1-1: random space-area character image add mode”, “1-2:follower add mode”, “1-3: mirror-reversed character image add mode,“2-1: object image select/replace mode” and “2-2: multiple imagepickup/replace mode”, and then sends a mode No. representing theselected mode to the image delivery server 110 (step TA6).

The digital camera 132 then starts to input a series of picked-up sceneimages through the image input unit 127 a to the terminal 120, whichthen sends the series of picked-up scene images to the image deliveryserver 110 (step TA7).

When the image delivery server 110 receives the mode No. representingthe selected mode from the terminal 120 (step SB7) and the series ofpicked-up scene images from the terminal 120 (step SB8), it performs aprocess for extracting the target image of FIG. 30 included in theserver process (step SBC).

In this target image extracting process, the series of picked-up sceneimages 12 hn received from the terminal 120 at predetermined timings arestored in the pickup scene memory 112 h of the work area (step SC1).

Thereupon, a series of successive posing object images Hn indicating theseries of successively changing acts of each object are extracted fromthe series of picked-up scene images 12 hn stored in the picked-up sceneimage memory 112 h (step SC2) and stored in the image processing workarea 12 i (step SC3).

That is, when the respective object images Hn contained in the picked-upscene image 12 hn are extracted as separate blocks from the picked-upscene image 12 hn and stored in the image processing work area 12 i(step SBC), a picked-up scene image 12 hn which contains the respectiveextracted character images Hn in broken-lined enclosures M is displayedon the display unit 118, for example, as shown in FIG. 11(A), 12(A) or13(A) (step SB9).

Thereupon, it is determined whether there are a plurality of blocks ofobject image Hn in the picked-up scene image 12 hn displayed on thedisplay unit 118 (step SB10).

If the image delivery server 110 determines that there are no pluralityof blocks of object image Hn in the picked-up scene image 12 hn, asingle object image Hn extracted from the picked-up scene image 12 hn isset as a target image of imitation (step SB10→SB11)

When the image delivery server 110 determines that there are a pluralityof blocks of object image Hn in the picked-up scene image 12 hn, theimage delivery server 110 sends the terminal 120 a request that the userspecify an object image block to be imitated (step SB10→SB12).

When the terminal 120 receives this request from the server 110 anddisplays it (step TA8), the user manipulates the input device 126 tospecify a target image of imitation from among the plurality of blocksof object image Hn and gives notice of the specified image to the imagedelivery server 110 (step TA9).

Then, the image delivery server 110 receives this notice from theterminal (step SB13).

The image delivery server 110 then extracts the specified object imageHn from the picked-up scene image 12 hn received from the terminal 120and sets it as a target image of imitation. Then, the image deliveryserver 110 shifts its process to a corresponding-animation imageacquiring process included in the server process of FIG. 31 (step SBD).

In this corresponding-animation image acquiring process, the imagedelivery server 110 first evaluates a similarity in pose of each of themodel images striking different poses stored in the model pose imagememory 112 b to the object image Hn extracted from the picked-up sceneimage 12 hn and set as the target image of imitation (step SDC). Theimage delivery server 110 then sends the terminal 120 a specifiedcharacter image A corresponding to a pose No. of the model image bearingthe highest pose similarity to the object image (step SD1).

When the terminal 120 receives that specified character image A from theimage delivery system 110 and displays it (step TA1), the userdetermines whether the displayed specified posing character image is“OK” or “next display” (display of a next candidate) should be clickedand then sends notice of this determination to the image delivery server110 (step TA11).

When the image delivery server 110 receives the notice of the selectionfrom the terminal 120 and determines that the terminal 120 has selected“the displayed specified posing character image is OK based upon thepresent evaluated similarity”, the image delivery server 110 registersthe specified character image (identified by its pose No.) as ananimation image corresponding to the target image of imitation (stepSD2→SD3).

When the image delivery server 110 receives the notice of the selectionfrom the terminal 120 and determines that the “next display” for theposing image of the received specified character is selected based uponthe present similarity evaluation, the image delivery server 110 sendsthe terminal 120 another specified character image A corresponding to apose No. of a model image bearing a second highest pose similarity. Thisis repeated each time the user's selection of the “next display” isreceived from the terminal 120 as long as there are posing characterimages corresponding to the posing model images (step SD2→SD4).

When the terminal 120 receives a posing character image corresponding toa model image bearing a desired pose similarity received from the imagedelivery server 110 and displays the posing character image, and theuser manipulates the input unit 126 to send “OK” for the displayedcharacter image to the image delivery server 110 (steps TA10-TA12), theimage delivery server 110 registers the OK'd character image (identifiedby its pose No.) as an animation image corresponding to the target imageof imitation (step SD2→SD3).

As described above, in the corresponding-animation image acquiringprocess by the image delivery server 110, a similarity in pose of amodel image representing each pose of a basic model of a person to theobject image Hn as the target image of imitation extracted from thepicked-up scene image 12 hn by the terminal 120 is evaluated, and aspecified character image to be used as an added one or replacementimitating the pose of the object image Hn is determined.

In this case, compared with the case in which a similarity in pose ofeach of the various posing images of a specified character to the objectimage Hn as the target image of imitation is directly evaluated tothereby determine a character image to be used as an added one or as areplacement, the corresponding-animation image acquiring process inwhich a similarity in pose of each model image to the object image Hn asthe target image of imitation is evaluated and a character image to beused as an added one or a replacement is determined from the pose No. ofthe selected model image is more advantageous in that the use of ahuman-figure model makes it easy to obtain a higher similarity in poseto the object image and hence a character image bearing a higher posesimilarity (imitation).

While the similarity evaluating process of FIG. 18 has been performed bythe image output apparatus of the first embodiment, a similar similarityevaluating process is performed by the image delivery server in thesecond embodiment. Thus, in the second embodiment the similarityevaluating process will be described by applying FIG. 18.

FIG. 18 is a flowchart of a process for evaluating a similarity in poseof an animation image to a target image of imitation in thecorresponding-animation image acquiring process included in the serverprocess to be performed by the image delivery server 110 of the imagedelivery system.

FIG. 32 illustrates examples of evaluating a similarity in pose of amodel character image to a target image of imitation in thecorresponding animation image acquiring process to be performed by theimage delivery server 110.

As shown in FIG. 32(A), a model image, for example, with a first poseNo. is read out (step C1) and adjusted in size to the target image ofimitation H1 of FIG. 32(X) (step C2).

Then, the target image H1 and the size-adjusted model image aresuperposed on each other, and a similarity in pose (%) between them isevaluated and stored as the similarity in pose of the model image to thetarget image H1 (step C3).

It is then determined whether the model image is asymmetrical (step C4).When it is determined that the model image is not asymmetrical, forexample, as shown by 12 b(1)A in FIG. 32(A) (step C4, “No.”), a nextpose No. model image is read out (step C8→C9) and a similarity in poseof this model image to the target image of imitation H1 is evaluated. Aslong as a further next model image is not asymmetrical, such similarityevaluating process is repeated (steps C2-C4).

If it is determined that a model image under similarity evaluation isasymmetrical for example, as shown by a pose No. 19 model image 12 b 19of FIG. 32(B), the model image is mirror reversed (step C4→C5). Then, asshown in FIG. 32(C), 1 the mirror-reversed model image 12 b(19)R isadjusted in size (lengthwise and widthwise) to the target image ofimitation (step C6).

Then, the target image H1 and the mirror-reversed model image 12 b(19) Rare superimposed over each other. A similarity in pose (%) between theseimages is evaluated and stored as the similarity in pose of themirror-reversed model image to the target image H1 (step C7).

Then, as long as there are still model image with respective pose Nos.(step C8), they are sequentially read out (step C9), and a similarity inpose of a respective one of the model images to the target image ofimitation is evaluated repeatedly (step C2-C7).

In this example, when, for example as shown in FIG. 32, the similarityin pose of the model image to the object (target) image H1 is evaluated,the model image 12 b(19) with pose No. 19 is evaluated as bearing thehighest similarity in pose to the target image H1.

As described above, when a specified character image imitating theobject image Hn is determined as the target image of imitation extractedfrom the picked-up scene image 12 hn in the corresponding-animationimage acquiring process (step SBD) described with respect to FIGS. 31,18 and 32, it is determined whether the image compose mode specified byand received from the terminal 120 is the “add/compose mode” orotherwise the “replace/compose mode” (step SB14).

If the image compose mode specified by the terminal 120 is the“add/compose mode” and the 1-1: random space-area character image addmode” (step SB14→SB15, “1-1”), the specified character image imitatingthe pose of the target image H1 as the target image of imitationacquired in the corresponding-animation image acquiring process (FIG.31) is added in the picked-up scene image 12 hn to a random position inits space-area except in a place from which the object image block isextracted, for example, as in FIG. 10 (step SB17 a). A resultingcomposite image is then sent to the terminal 120 (step SB22).

The terminal 120 then receives the composite image from the imagedelivery server 110 and displays it (step TA13). When the user at theterminal 120 performs a process for moving the character image from itsresent position to any specified different position in the compositepicked-up scene image 12 hn, information on its movement is then sent tothe image delivery server 110 (step TA14).

When the image delivery server 110 receives this information from theterminal 120 (step SB23→SB24), the image delivery server 110 moves thecharacter image to the different position based upon the information inthe space-area of the picked-up scene image 12 hn (step SB25) and thensends a resulting composite image to the terminal 120 (step SB26).

Thereupon, the terminal 120 receives and displays this composite imagefrom the image delivery server 110 (step TA15→TA16).

That is, the image delivery server 110 sends the terminal 120 data onthe composite image in which the specified character image imitating thepose of the specified object image H1 and created in the randomspace-area character image add mode is added to a random position in thespace area of the picked-up scene image 12 hn.

When the image delivery server 110 determines that the image composemode specified by the terminal 120 is the “add/compose mode” and the“1-2: follower add mode” (step SB14→SB15, “1-2”), it adds a specifiedcharacter image imitating a waling pose of the object image (targetimage) in the picked-up scene image 12 hn acquired in thecorresponding-animation image acquiring process (FIG. 31) at a positionin the picked-up scene image 12 hn where the specified character imagefollows the walking object image (target image) H at a given distancefrom this object image, or at a position in the picked-up scene images12 hn where the object (target) image assumed a certain time ago, as inFIG. 9 (step SB17 b). This image delivery server 110 then sends aresulting composite image to the terminal 120 (step SB22).

When the terminal 120 receives the composite image from the imagedelivery server 110 and displays it there (step TA13), and the user ofthe terminal 120 manually performs a process for moving the specifiedcharacter image as the follower in the composite image from its presentposition to any specified different position, information on itsmovement is sent to the image delivery server 110 (step TA14).

When the image delivery server 110 receives this information from theterminal 120 (step SB23→SB24), the image delivery server 110 again addsthe specified character image as the follower to the different positionin the picked-up scene image 12 hn (step SB25) and sends a resultingcomposite image to the terminal 120 (step SB26).

Then, the terminal 120 receives this composite image from the imagedelivery server 110 and displays it (step TA15→TA16).

That is, the image delivery server 110 sends the terminal 120 data on acomposite image in which the specified character image imitating thepose of the specified object image H and created in the follower addmode is added to the follower's position after the object image H.

When the image delivery server 110 determines that the image composemode specified by the terminal 120 includes the “add/compose mode” andthe “1-3: mirror-reversed character image add mode” (step SB14→SB15,“1-3”), the image delivery server 110 mirror reverses a “rabbit”character image 12 d{circle over (19)} imitating the dancing pose of anobject image (target image) H in the scene image 12 g 3, adds to aposition in the scene image 12 g 3 where the mirror-reversed “rabbit”character image links with the target image H (step SB17 c), and thensends a resulting composite scene image 12 g 3′ to the terminal 120(step SB22).

When the terminal 120 receives the composite image 12 g 3′ from thescene image delivery server 110 and displays it (step TA13), and theuser manually performs a process for moving the reversed “rabbit”character image 12 d{circle over (19)}R (R means reversal) in thepicked-up scene image 12 g 3′ from its present position to any specifieddifferent position, information on its movement is delivered to theimage delivery server 110 (step TA14).

When the image delivery server 110 receives the movement informationfrom the terminal 120 (step SB23→SB24), the image delivery server 110again adds the mirror-reversed “rabbit” character image to the differentposition based upon the movement information in the scene image 12 g 3′(step SB25), and a resulting composite scene image is then sent to theterminal 120 (step SB26).

Then, the terminal 120 receives this composite scene image from theimage delivery server 110 and displays it (step TA15→TA16).

That is, the image delivery server 110 sends the terminal 120 data on ascene image 12 g 3 in which the mirror-reversed “rabbit” character imageimitating the pose of the dancing object image H and created in themirror-reversed character image add mode is added to the position wherethe mirror-reversed “rabbit” character image links with the object imageH, for example, as in FIG. 11(D).

If the image delivery server 110 determines that the image compose modespecified by the terminal 120 includes the “replace/compose mode” andthe “2-1: object image select/replace mode” (step SB14→SB16, “2-1”), theimage delivery server 110 replaces the specified object image (targetimage) H1 with a “rabbit” character image 12 d{circle over (1)}imitating the pose of the object image H1 as the target image ofimitation specified in the picked-up scene image 12 g 5 acquired in thecorresponding-animation image acquiring process (FIG. 31) (step SB18),and then sends a resulting composite image to the terminal 120, forexample, as in FIG. 13 (step SB22).

In this case, three object images, that is, the target object image H1specified in the picked-up scene image 12 g 5, and other two objectimages H2 and H3 are cut out from the picked-up scene image 12 g 5, anda picked-up background image 12 g 5′ is produced. The “rabbit” characterimage 12 d{circle over (1)} similar in pose to the specified targetobject image H1, the remaining two object images H2 and H3 cut out fromthe scene image 12 g 5 are then added to the respective relevantpositions in the picked-up background image 12 g 5′ in decreasing orderof the sensed image pickup distances to the respective objects concernedto thereby provide a resulting composite image.

Without producing the remaining background image 12 g 5′, the “rabbit”character image 12 d{circle over (1)} similar in pose to the targetimage H1 and the other two images H2 and H3 may be added to the theirrelevant positions in the picked-up scene image 12 g 5 in decreasingorder of the sensed image pickup distances to the respective objectsconcerned.

When the terminal 120 receives the composite image from the imagedelivery server 110 and displays it (step TA13), and manually performsthe process for moving the “rabbit” character image 12 d{circle over(1)} added in place of the target image H1 in the picked-up scene image12 g 5″ from its present position to any specified different position,information on its movement is then sent to the image delivery server110 (step TA14).

When the image delivery server 110 receives this information (stepSB23→SB24), it readds the “rabbit” character image 12 d{circle over (1)}to the different position based upon the movement information in thepicked-up scene image 12 g 5″ in place of the target character image H1(step SB25), and sends a resulting composite image to the terminal 120(step SB26).

Thereupon, the terminal 120 receives this composite image from the imagedelivery server 110 and displays it (step TA15→TA16).

That is, the image delivery server 110 sends the terminal 120 data on acomposite image created in the object image select/replace mode and inwhich the object image H1 is replaced with the “rabbit” character imageimitating the pose “Say cheese” of the object image H1, as in FIG.13(G).

If the image delivery server 110 determines that the image compose modespecified by the terminal 120 includes the “replace/compose” mode andthe “2-2: the multiple image pick-up/replace mode” (step SB14→SB16,“2-2”), the position of the target object image H1 in a first picked-upscene image 12 h 2 is stored as the position where the imitating imagereplaces the target object image H1, for example, as shown in FIG. 25.(step SB16→SB19).

When the image delivery server 110 acquires a second picked-up sceneimage 12 h 2′ (step SB20), it adds a “bear” character image 12 e{circleover (11)}N (in this case, a special “night bear” character image)imitating the pose of the object image H1 acquired in thecorresponding-animation image acquiring process (FIG. 31) to a positionin the second picked-up scene image 12 h 2′ corresponding to the storedposition of the object image (target image) H1 in the first picked-upscene image 12 h 2 such that the “bear” character image 12 e{circle over(11)}N waves good-by to the object image H2 (step SB21) and a resultingcomposite image 12 h 2″ is then sent to the terminal 120 (step SB22).

When the terminal 120 receives and displays the composite image 12 h 2″from the image delivery server 110 (step TA13) and manually performs theprocess for moving the “night bear” character image 12 e{circle over(11)}n in the composite image 12 h 2″ from its present position to anyspecified different position in the composite image 12 h 2″, informationon its movement is sent to the image delivery server 110 (step TA14).

When the image delivery server 110 receives this information (stepSB23→SB24), it readds the “bear” character image 12 e{circle over(11)}N′ to the different position in the second picked-up scene image 12h 2′ based upon the received movement information (step SB25), and aresulting composite image is then sent to the terminal 120 (step SB26).

Then, the terminal 120 receives this composite image from the imagedelivery server 110 and displays it (step TA15→TA16).

That is, the image delivery server 110 sends the terminal 120 thecomposite image 12 h 2″ created in the multiple image pickup/replacemode in which the “bear” character image (particular “night” characterimage) imitating the “Good-by waving” pose of the object image H1 in thefirst picked-up scene image 12 h 2 is combined with the object image H2in the second picked-up scene image 12 h 2′. A resulting composite imageis then delivered to the terminal 120.

The terminal 120 receives and displays this composite image. When theuser of the terminal 120 desires to send the composite image displayedon the terminal 120 to a specified different terminal, the user inputsan address of the different terminal via the input unit 126 and thensends the composite image to the image delivery server 110 (stepTA17→TA18).

When the image delivery server 110 receives the address of the differentterminal from the terminal 120 (step SB27), the image delivery server110 sends the composite image to the different terminal of that address(step SB28), calculates a charge for the composite image-creating andsending services provided this time to the requester user and thespecified different terminal and sends the requester terminal 120 thecontents of the charge (step SB29).

The requester terminal 120 then receives the contents of the charge fromthe image delivery server 110 and displays it on the display 130 (stepTA19).

When the terminal 120 inputs and sends no address of the specifieddifferent terminal (step TA17→No), the image delivery server 110calculates a charge for only the composite image creating and deliveringservices provided to the requester terminal this time and sends therequesting terminal 120 the contents of the charge (step S27→SB30).

Thereupon, the requester terminal 120 receives and displays the contentsof the charge from the image delivery server 110 and displays it on thedisplay 30 (step TA20).

The image delivery server 110 sends a corresponding bill to a bankaccount of the requester terminal registered in the requester terminalinformation memory 112 g (step SB 31).

Therefore, according to the image delivering function of the imagedelivery system of the second embodiment, when the scene image picked-upby the terminal 120 is sent to the image delivery server 110, the imagedelivery server 110 extracts an object image from the received sceneimage, acquires any specified type of character image similar in pose tothe extracted object image from among the character 1 “rabbit” poseimage memory 12 d, the character 2 “bear” pose image memory 12 e, andthe character 3 “cat” pose image memory 12 f, replaces the object imagein the picked-up scene image with the acquired specified type ofcharacter image in the picked-up scene image or adds the requiredcharacter image to the scene image to thereby provide a resultingcomposite image, and sends this composite image to the terminal 120.Thus, a very delightful composite scene image in which an animationimage imitating the pose of the target object in the original sceneimage is used as an added one or otherwise as one replacing the targetobject is created and delivered to the terminal without creating anddelivering a scene image to which a fixed animation image is simplyadded in the image pickup frame.

By using the image delivery server 110 in the image delivery system ofthe second embodiment, various services are provided which includedelivery of a mail with a composite scene image including an added orreplacing character image imitating the pose of an object image in thepicked-up scene image, pay composition and delivery of a composite sceneimage in which a character image imitating the pose of a target image ina picked-up original scene image is used as an added or replacing one inthe original scene image to a respective one of image-pickup type mobileterminals 120B, 120C, 120D such as a cellular phone terminal 120A with adigital camera, pay or free creation and delivery of a composite sceneimage in which a particular character image imitating the pose of theobject image in a picked-up scene image is added to, or otherwisereplaces the object image, for publicizing the particular character on aparticular home page, and pay creation and delivery of a picked-up sceneimage in which a popular character image imitating the pose of an objectimage in a picked-up scene image is added to a digital camera terminal120D connected via a short-distance wireless communication unit 116 onlyin a specified place such as an amusement spot.

As described above, in the image delivery system of the secondembodiment, an image picked up by the terminal 120 is sent to the imagedelivery server 110. In the image delivery server 110, an object imageis extracted from the picked-up scene image, a character image isacquired which is similar in pose to the extracted object image, thecharacter image is added to the scene image or replaces the object imagein the scene image. A resulting composite image is then delivered to theterminal 120 and a bill for this service is sent to the terminal 120. Incontract, as will be described in the next third embodiment, theterminal 120 extracts an object image from the picked-up scene image,determines a pose No. of a model image similar in pose to the extractedobject image, and requests the image delivery server 110 to deliver aspecified character image with the determined pose No. The imagedelivery server 110 only delivers to the terminal 120 the characterimage with the determined pose No., which was requested by the terminal120, and bills the terminal for this service. The terminal 120 furtheradds the delivered character image to the scene image or otherwisereplaces an object image in the scene image with the delivered characterimage to thereby provide a resulting composite image.

Third Embodiment

The electronic circuits of the image delivery server 110 and terminals120 (120A-120D) of the image delivery system as the third embodiment ofthe present invention are substantially similar to those of the secondembodiment described with reference to FIGS. 23 and 24, excluding thefollowing points:

In the image delivering system of the third embodiment, the model imagememory 112 b and the size-adjusted model pose image memory 112 c are notprovided in the image delivery server 110, but in the terminal 120.

FIG. 33 is a flowchart of an image process (part 1) to be performed bythe terminal 120 in the image delivery system of the third embodiment.

FIG. 34 is a flowchart of an image process (part 2) continued from theimage process (part 1) of FIG. 33.

FIG. 35 is a flowchart of a server process to be performed by the imagedelivery server 110 of the third embodiment.

First, the terminal 120 (120A, 120B or 120C) responds to themanipulation of the input unit 126 to connect via the transmissioncontroller 125 or short-distance wireless communication unit 128 to theimage delivery server 110 on the Internet N or in a short-distancewireless communication range and then to send the user's ID and passwordto the image delivery server 110 (step TF1).

When the image delivery server 110 receives the ID and password from theterminal 120 (120A, 120B or 120C) via the transmission unit 115 throughthe Internet N or the terminal (digital camera) 120D via theshort-distance wireless communication unit 116 (step SJ1), it determineswhether the terminal 120 is a fill member or not depending on whetherthe terminal ID and password can find their matches among the registeredIDs and passwords of the member terminals in the requester terminalinformation memory 112 g of the memory 112 (step SJ2).

When it is determined that the ID and password of the terminal 120 cannot find their matches among the registered IDs and passwords of themember terminals and that the terminal is not a full member, theconnection of the terminal 120 to the image delivery server 110 isinterrupted and an error process is performed (step SJ2, “No.”).

When the connection of the terminal 120 to the image delivery server 110is not interrupted over a predetermined time after the terminal 120 hassent its ID and password to the image delivery server 110 (step TF1),data representing the present position of the terminal 120 (in the caseof the cellular phone 120A, data representing the position of its basestation concerned) sensed by GPS 129 is sent from the terminal 120 tothe image delivery server 110 (step TF2).

When the image delivery server 110 receives from the terminal 120 datarepresenting its present position (step SJ3), it determines whether theterminal 120 is at present in a predetermined particular place (forexample, a souvenir picture pickup place in a theme park or touristspot) or whether the present time is in a specified time zone (forexample, at night) (step SJ4).

If the image delivery server 110 determines that the terminal 120 underconnection is at present in the predetermined particular place or thatthe present time is in the specified time zone (step SJ4), a specialcharacter corresponding to the particular place (for example, in thecase of a theme park, a popular character there) or a special charactercorresponding to the specified time zone (for example, in the case of atnight, a “bear” character sitting on the sickle moon) is determined asthe special character corresponding to the target object, and the nameof this character is then sent to the terminal 120 (step SJ5).

When the image delivery server 110 determines that the terminal 120under connection is not in the predetermined place, or that the presenttime is not in the specified time zone (step SJ4), the image deliveryserver 110 waits for reception of a specified type of character from theterminal 120 (step SJ6).

If the terminal 120 receives the name of the special characterdetermined and sent by the image delivery server 110 because theterminal 120 is in the particular place or the present time is in thespecified time zone (step TF3), the special character name is deliveredto and displayed by the display unit 130 (step TF4).

When the terminal 120 does not receive the name of the special characterfrom the image delivery server 110 because the terminal 120 is not inthe particular place or the present time is not in the specified timezone (step TF3), the user of the terminal 120 manipulates the input unit126 to thereby specify the type of a character (“rabbit”, “bear” or“cat”) to be used as an added one or a replacement in the picked-upscene image and sends the specified type of a character to the imagedelivery server 110 (step TF5).

Thereupon, the picked-up scene images start to be input from the digitalcamera 132 via the image input unit 127 a to the terminal 120 (stepTF6), and the process proceeds to the target image extracting process(TFG) of FIG. 36 included in the image process by the terminal 120.

In this target image extracting process, the series of picked-up sceneimages inputted through the image input unit 127 a into the terminal 120at predetermined timings are stored in the work area 122 d of the memory122 (step TG1).

Thereupon, a series of successive object images Hn indicating therespective acts of each object are extracted as separate blocks from theseries of picked-up scene images stored in the image processing workarea 122 d (step TG2) and stored as the respective blocks in the workarea 122 d (step TG3).

That is, when the respective object images Hn contained in the picked-upscene image are extracted as separate blocks from the picked-up sceneimages and stored in the image processing work area 122 d (step TFG), apicked-up scene image which contains the respective extracted characterimages Hn in their broken-lined enclosures M is displayed on the displayunit 130, for example, as in FIG. 11(A), 12(A) or 13(A) (step TF7).

Thereupon, it is then determined whether there are a plurality of blocksof object image Hn in the picked-up scene image displayed on the displayunit 130 (step TF8).

If it is determined that there are no plurality of blocks of objectimage Hn, a single object image Hn extracted from the picked-up sceneimage is set as a target image of imitation (step TF8→TF9)

If it is determined that there are a plurality of blocks of object imageHn, an object image Hn is specified as a target image of imitation fromamong those blocks of object image Hn by the user's manipulation of theinput unit 126 (step TF8→TF10)

When the object image Hn is extracted from the picked-up scene image andspecified as the target image of imitation Hn, the process proceeds to apose No. determining process of FIG. 37 (step TFI).

In this pose No. determining process, a similarity in pose of each ofthe differently posing model images stored in the model pose imagememory 122 b to the object image Hn extracted from the picked-up sceneimage and set as the target image of imitation is evaluated (step TIK),and a posing model image bearing the highest pose similarity isdisplayed on the display unit 130 (step TI1).

When this displayed model image is selected by the user at the inputunit 126, the pose No. of this selected model image is registered as theone corresponding to the target image of imitation (step TI2→TI3).

When the “next display” is clicked by the user's manipulation on theinput unit 126 in a state where a model image corresponding to the poseNo. representing the highest similarity in pose to the target image ofimitation is displayed, a model image with a pose No. representing asecond highest similarity in pose to the target image is displayed. Aslong as there are still a plurality of model images having differentsimilarities, a model image bearing a next lower pose similarity isdisplayed each time the “next display” is clicked. (step TI2→TI4).

When a model image with a pose No. representing a desired posesimilarity is displayed and selected by the user's manipulation on theinput unit 126, the pose No. of this model image is registered as thatcorresponding to the target image of imitation (step TI2→TI3).

As described above, in the pose No. determining process, the posesimilarity of each model image (striking a respective pose) representinga basic model of a person to the object image Hn as the target image ofimitation extracted from the picked-up scene image is evaluated, and thepose No. of a model image imitating the pose of the object image Hn isdetermined based upon the evaluation of the similarity.

FIG. 38 is a flowchart of evaluating a similarity in pose of a modelimage to the target image of imitation in the pose No. determiningprocess included in the image process to be performed by the terminal120 in the image delivering system of the third embodiment.

For example, as shown in FIG. 32(A), a model image with a first pose No.is read out (step TK1) and adjusted in size to the target image ofimitation H1 of FIG. 32(X) (step TK2).

Then, the target image H1 and the size-adjusted model image aresuperposed on each other, and a similarity in pose (%) between them isevaluated and stored as the similarity in pose of the model image to thetarget image H1 (step TK3).

It is then determined whether the model image with the pose No. ofinterest is asymmetrical (step TK4). When it is determined that themodel image is not asymmetrical, for example, as shown by 12 bA(1) inFIG. 32(A) (step TK4, “No.”), a next pose No. model image is read out(step TK8→TK9) and a similarity in pose of this model image to thetarget image of imitation H1 is evaluated likewise. As long as a furthernext model image is not asymmetrical, such similarity evaluating processis repeated (steps TK2-TK4).

If it is determined that the model image of similarity evaluation isasymmetrical, for example, as shown by a pose No. 19 model image 12 b 19of FIG. 32(B), the model image is mirror reversed (step TK4→TK5). Then,as shown in FIG. 32(C), 1 the mirror-reversed model image 12 b(19)R isadjusted in size (lengthwise and widthwise) to the target image ofimitation (step TK6).

Then, the target image H1 and the mirror-reversed model image 12 b(19)Rare superposed on each other. A similarity in pose between these images(%) is evaluated and stored as the pose similarity of themirror-reversed model image to the target image H1 (step TK7).

Then, as long as there are further model image of different pose Nos.(step TK8), they are sequentially read out (step TK9), and a similarityin pose of a respective one of the model images to the target image ofimitation is repeatedly evaluated (step TK2-TK7).

In this example, when for example, as shown in FIG. 32, the similarityin pose of the model image to the object (target) image H1 is evaluated,the model image 12 b(19) with pose No. 19 is evaluated as bearing thehighest similarity in pose to the target image H1.

As described above, when a pose No. of a model image imitating the poseof the object image Hn as the target image of imitation extracted fromthe picked-up scene image is determined in the pose No. determiningprocess (step TFI) described with respect to FIGS. 37, 38 and 32, arequest for delivery of a specified character image striking a pose forthe determined pose No. is then sent to the image delivery server 110(step TF11).

When the image delivery server 110 receives the request for delivery ofa character image from the terminal 120 (step SJ7), the image deliveryserver 110 selectively reads a character image with the determined poseNo. from an appropriate one of the character 1 “rabbit” pose imagememory 112 d, the character 2 “bear” pose image memory 112 e, and thecharacter 3 “cat” pose image memory 112 f included in the memory 112 anddelivers the read character image to the requester terminal 120 (stepSJ8).

Thereupon, the image delivery server 110 calculates the charge for thedelivery service provided to the requester user and sends notice of thecontents of the charge to the requester terminal 120 (step SJ9), and acorresponding bill is sent to a bank account of the requester terminalregistered in the requester terminal information memory 112 g (stepSJ10).

When the requester terminal 120 receives the character image with thedetermined pose No. from the image delivery server 110 (step TF12),either the add/compose mode that adds the received character image tothe picked-up scene image or the replace/compose mode that replaces thetarget image H in the picked-up scene image with the received characterimage is selected from among the “1-1: random space-area character imageadd mode”, “1-2: follower add mode”, “1-3: mirror-reversed characterimage add mode”, “2-1: object image select/replace mode” or “2-2:multiple image pickup/replace mode” (step TF13).

In the flowchart of the image process performed by the terminal 120 inFIGS. 33 and 34, routines corresponding to the replace/compose mode“2-1: object image select/replace mode” and “2-2: multiple imagepickup/replace mode” are omitted. The omitted routines include the sameprocess as are expressed by the steps SB16, SB18-SB21 of FIGS. 28 and 29and are selectively performed along with the process of steps TF14 andTF15 a-TF15 c in the add/compose mode which will be described later.

Therefore, only the process to be performed when the add/compose mode isselected will be described herein and the process to be performed whenthe replace/compose mode is selected will be omitted.

If it is determined in step TF13 that the specified-image compose modeis the “add/compose mode” and the “1-1: random space-area characterimage add mode” (step F14, “1-1”), the specified character imageimitating the pose of the target image Hn as the target image ofimitation delivered by the image delivery server 110 is added in thepicked-up scene image 12 hn to a random position in its space-areaexcept in the place from which the object image block is extracted. Aresulting composite image is then displayed (steps TF15 a, TF16).

Then, when the process for moving the character image in the compositeimage from its present position to any specified different position isperformed in the composite image (step TF17→step TF18), the characterimage is moved to the different position in the composite image, andanother resulting composite image is then displayed (TF19-TF20).

When it is determined that the specified-image compose mode includes the“add/compose mode” and the “1-2: follower add mode” (step TF14, “1-2”),a specified character image delivered by the image delivery server 110and imitating a walking pose of the object image (target image) in thepicked-up scene image is added to a position in the latest scene imagecorresponding to the position on a relevant one of the series ofpicked-up scene images 12 hn where the walking object image (targetimage) H assumed a certain time ago to thereby provide a resultingcomposite image, as in FIG. 9(D) (steps TH15 b, TF16).

When the process for moving the specified character image as thefollower in the composite image from its present position to anyspecified different position in the composite image is performedmanually (step TF17 to TF18), the specified character image is moved tothe different position in the composite image and another resultingcomposite image is then displayed (steps TF19, TF20).

When it is determined that the specified-image compose mode includes the“add/compose mode” and the “1-3: reversed-character image add mode”(step F14, “1-3”), a “rabbit” character image 12 d{circle over (19)}imitating the dancing pose of an object image (target image) H in thescene image 12 g 3 is mirror reversed, the mirror-reversed “rabbit”character image is added at a position in the scene image 12 g 3 wherethe mirror reversed “rabbit” character image links with the target imageH for dancing purposes, and then a resulting composite image 12 g 3′ isdisplayed, as shown in FIG. 11(D) (step TF15 c→TF16).

When the process for moving the reversed “rabbit” character image 12d{circle over (19)}R in the composite image 12 g 3′ from its presentposition to any specified different position is manually performed (stepTF17 to step TF18), the mirror reversed “rabbit” character image ismoved to the different position in the composite image 12 g 3′, andanother resulting composite image is then displayed (steps TF19, TF20).

Then, the terminal 120 receives notice of the contents of the charge forthe use of the server service by the user from the image delivery server110 and displays it on the display unit 130 (step TF21).

Therefore, according to the image delivering function of the imagedelivery system of the third embodiment, when the terminal 120 picks upan image of a scene containing objects, an object image is extractedfrom the picked-up scene image. A pose No. is determined from a modelimage similar in pose to the extracted object image. The terminal 120requests delivery of a posing character image with the determined poseNo. and a specified type of the character image from the image deliveryserver 110. Thereupon, the image delivery server 110 reads a characterimage with the specified pose No. and type from an appropriate one ofthe character 1 “rabbit” pose image memory 112 d, the character 2 “bear”pose image memory 112 e, and the character 3 “cat” pose image memory 112f, and delivers the read character image to the terminal 120, whichreplaces the object image in the picked up scene image with the receivedcharacter image similar in pose to the object or adds the receivedcharacter image to the scene image, and displays a resulting compositeimage. That is, the user at the terminal 120 does not simply add a fixedanimation image to the pickup frame for delighting purposes, butacquires an animation image similar in pose to the object image in thescene image from the image delivery server 110 and replaces the objectimage in the picked-up scene image with the animation image or adds theanimation image to the picked-up scene image to thereby produce a verydelightful composite image containing the animation image.

The image delivery server 110 in the image delivery system of the thirdembodiment is capable of providing image-pickup type mobile terminals120B, 120C, 120D such as a cellular phone terminal 120A, which includesa digital camera, with pay delivery of a character image imitating apose of an object image in each picked-up scene image, pay or freedelivery of a particular character image imitating a pose of an objectimage in a picked-up scene image for publicizing the particularcharacter on a particular home page, and pay delivery of a popularcharacter image imitating a pose of an object image in a picked-up sceneimage to a digital camera terminal 120D via a short-distance wirelesscommunication unit 116 only in a specified place such as an amusementspot, etc.

While in the respective embodiments a character image similar in pose tothe object image extracted from the picked-up scene image is selectivelyobtained and used as an added one or a replacement in the picked-upscene image to thereby provide a resulting composite image, face imageshaving various expressions and person images actually picked-up may bestored beforehand and used as an added one or a replacement imitating apose of an object image extracted from the picked-up scene image tothereby output a resulting composite image.

In the respective above embodiments the picked-up scene image as a stillimage has been illustrated. On the other hand, when the picked-up sceneimages include a series of animation images inputted, object imageschanging in time series relative to the background image of the seriesof dynamic images inputted may be sequentially extracted from the seriesof animation images. Character images (FIG. 39) similar in pose to thesequentially extracted object images may then be selectively acquired,delivered, or added to the picked-up dynamic images or replace theobject images in the respective animation images to thereby provide aseries of resulting composite animation images, which will be thendelivered.

FIG. 39 illustrates animation character images striking various posesstored in the character “rabbit” animation pose image memory 112 dM inthe image delivery server 110.

By utilizing the image processing function of the image delivery server110 or terminal 120 in the respective embodiments, a prepared desiredcharacter image similar in pose to a target image of imitation thatincludes an appearing character specified and extracted, for example,from an animation image in a game, or otherwise an actually picked-upperson image may be acquired and delivered to the requester terminal orreplace the character appearing in the animation image to therebydeliver a resulting composite image to the requester terminal.

As will be described in a next fourth embodiment, object images strikingvarious poses picked up by the user, or character images strikingvarious poses appearing in a game downloaded from game servers (game HP)on the Internet N may be recorded in memories similar to the character 1“rabbit” pose image memory 112 d, the character 2 “bear” pose imagememory 112 e, and the character 3 “cat” pose image memory 112 f of thememory 112 in the image delivery server 110. When a character imagesimilar in pose to the object image extracted from the picked-up scenemage in the second or third embodiment is acquired from the imagedelivery server 110, it may be selected from among the originallyregistered “rabbit”, “bear” and “cat” character images, and the personimages and game character images registered by the user.

Fourth Embodiment

FIG. 40 is a block diagram indicative of a whole composition of an imagedelivery system in the fourth embodiment.

This embodiment handles the user's pickup of object images strikingvarious poses, using a cellular phone 220A with a digital camera, andregistration of the object images as the user's original characterimages in a memory 212 of an image delivery server 210, or otherwiseregistration of game characters striking various poses downloaded from amemory 212A of a game server 210A as the user's original characters inthe memory 212 of the delivery server 210.

FIGS. 41(A) and 41(B) are flowcharts of the user's original characterimage registering processes to be performed in the cellular phone 220Aand the image delivery server 210, respectively, of the image deliverysystem in the fourth embodiment.

First, when the user's original character images are downloaded by andregistered on the cellular phone 220A with the digital camera from thegame server 210A on the Internet N, a homepage (HP) of the game server210A from which the character images are acquired is specified, andconnected to the cellular phone 220A (step TS1→TS2).

When the user specifies a game character to be set as his or heroriginal character image on the homepage (HP) of the game server 210A(step TS3), the various posing images of a specified game characterstored in the memory 212A of the game server 210A are downloaded (stepTS4).

Thereupon, the user ID (=123) of the cellular phone 220A and therespective posing character images downloaded from the game server 210Aare delivered to the image delivery server 210 (step TS5).

When the user picks up and registers the user's original characterimages using the cellular phone 220A without downloading similarcharacter images from the Internet N, the digital camera 232 picks up ascene image containing object images striking their respective poses(step TS1→TS7).

Thereupon, the cellular phone 220A extracts the character images asobject images excluding the background image in the picked-up sceneimage (step TS8) and stores them as corresponding-character imagestaking respective poses (step TS9).

Then, the cellular phone 220A then sends its terminal user's ID (=123)and the stored corresponding-images striking the respective poses to theimage delivery server 210 (step TS5).

When the image delivery server 210 receives the terminal user's ID andthe character images striking their respective poses from the cellularphone 220A (step SR1), it registers the character images as the onesstriking respective poses for the user's ID, apart from the respective“rabbit”, “bear” and “cat” character images (112 d-112 f), for example,of FIG. 23 in the memory 212 (step SR2).

When the image delivery server 210 selects and deliver a character imagesimilar in pose to the object image extracted from the picked-up sceneimage in the second or third embodiment, the image delivery server 210can select a character image from the registered “rabbit”, “bear” and“cat” character images, and the character images picked up andregistered by the user, and the game characters.

The various processes performed by the image delivery system describedwith reference to the respective embodiments, that is, the image processshown by the flowchart of FIGS. 26 and 27 performed by the terminal 120of the image delivery system of the second embodiment, the serverprocess performed by the image delivery server 110 shown by theflowcharts of FIGS. 28 and 29 in the image delivery system of the secondembodiment, the object image extracting process included in the serverprocess in the second embodiment shown by the flowchart of FIG. 30, thecorresponding-animation image acquiring process included in the serverprocess of the second embodiment shown by the flowchart of FIG. 31, theprocess of evaluating the similarity in pose of the character/modelimage to the target image of imitation included in thecorresponding-animation image acquiring process in the second embodimentshown by the flowchart of FIG. 18, the image process performed by theterminal 120 in the image delivery system of the third embodiment shownby the flowcharts of FIGS. 33 and 34, the server process of the imagedelivery server 110 in the image delivery system of the third embodimentshown by the flowchart of FIG. 35, the target image extracting processincluded in the image process of the third embodiment shown by theflowchart of FIG. 36, the pose number determining process included inthe image process of the third embodiment shown by the flowchart of FIG.37, the process of evaluating the similarity in pose of the model poseimage to the target image of imitation included in the pose numberdetermining process in the third embodiment shown by the flowchart ofFIG. 38, the user's character image registering process performed in thecellular phone 220A in the image delivery system of the fourthembodiment shown by the flowchart of FIG. 41, and the user's characterimage registering process performed by the image delivery server 210 maybe stored and distributed as computer-executable programs on memorycards (ROM cards, RAM cards, etc.), magnetic discs (floppy discs, harddiscs, etc.), optical discs (CD-ROMs, DVDs, etc.), and externalrecording mediums such as semiconductor memories 113 (123, 213, 223).Various computer terminals having communication functions with thecommunication networks (the Internet) N can read the programs stored onthe external recording mediums 113 (123, 213, 223) into the memories 112(122, 212, 222) using the recording medium readers 114 (124, 214, 224).Thus, operation of the computers may be controlled by the these programsto perform the image delivering function described with reference to therespective embodiments to thereby perform the processes similar to thosementioned above.

The program data to realize the respective processes can be transmittedthrough the communication network (the Internet) N in the form of aprogram code. The program data can be taken from the computer terminalsconnected to the communication network (the Internet) N to realize theimage delivering function.

The present invention is not limited to the ones implemented by therespective embodiments. Various changes and modification could be easilymade by those skilled in the art without departing from the spirit andscope of the invention.

1. A composite image output apparatus characterized by comprising: apicked-up scene image acquiring unit (17) which acquires a picked-upscene image representing a scene containing at least one object; atarget image extracting unit (11) which extracts a target imagerepresenting any particular one of the at least one object from thepicked-up scene image acquired by said acquiring unit; acorresponding-image obtaining unit (11) which obtains an imagecorresponding to the target image extracted by said extracting unit; andan image outputting unit (11, 20) which adds the image obtained by saidobtaining unit to the picked-up scene image acquired by said picked-upscene image acquiring unit and outputs a resulting composite image.
 2. Acomposite image output apparatus characterized by comprising: apicked-up scene image acquiring unit (17) which acquires a plurality ofpicked-up scene images, each representing a scene containing at leastone object; a target image extracting unit (11) which extracts a targetimage representing any particular one of the at least one object fromany selected one of the plurality of picked-up scene image acquired bysaid picked-up scene image acquiring unit; a corresponding-imageobtaining unit (11) which obtains an image corresponding to the targetimage extracted by said extracting unit; and an image outputting unit(11, 20) which replaces in the picked-up scene image the target imageextracted by said target image extracting unit with the image obtainedby said corresponding-image obtaining unit, adds the corresponding-imageto any selected one of the others of said plurality of picked-up sceneimages from which no target images are extracted by said target imageextracting unit, and outputs a resulting composite image.
 3. Thecomposite image output apparatus according to claim 1, characterized inthat said picked-up scene image acquiring unit acquires a series ofsuccessively changing picked-up scene images; and said target imageextracting unit extracts a series of successively changing images of anyparticular one of the at least one object from among the series ofsuccessively changing scene images acquired by said picked-up sceneimage acquiring unit.
 4. The composite image output apparatus accordingto claim 1, characterized in that said target image extracting unitcomprises: a unit (18) which measures a distance from the compositeimage output apparatus to each of the at least one object in the scene;and a unit (11) which extracts the target image in accordance with thedistance measured by said measuring unit from the composite image outputapparatus to each of the at least one object in the scene.
 5. Thecomposite image output apparatus according to claim 1, characterized inthat said corresponding-image obtaining unit comprises: storage means(12 d, 12 e, 12 f) having stored a plurality of images corresponding tothe target image; and means (11) for evaluating a similarity in pose ofeach of the images stored in said storage means to the target imageextracted by said target image extracting unit, means (11) for obtainingfrom said storage means an image bearing a specified similarity in poseto the target image evaluated by said similarity evaluating means. 6.The composite image output apparatus according to claim 1, characterizedin that said corresponding-image obtaining unit comprises: acorresponding-image storage unit (12 b,12 c,12 d, 12 e, 12 f) havingstored a plurality of model images and a plurality of imagescorresponding respectively to the plurality of model images incorresponding relationship; similarity evaluating means (11) forevaluating a similarity in pose of each of the model images stored insaid corresponding-image storage means to the target image extracted bysaid image extracting unit; and means (11) for obtaining from saidcorresponding-image storage means an image corresponding to a modelimage bearing a specified similarity in pose to the target imageevaluated by said similarity evaluating means.
 7. The composite imageoutput apparatus according to claim 1, characterized in that saidcorresponding-image obtaining unit comprises: corresponding-imagestorage means (12 d, 12 e, 12 f) having stored a plurality of groups ofdifferent images for a plurality of respective different types, eachimage corresponding to the target image; type specifying means (11) forspecifying any particular one of the plurality of different types forwhich different images of a relevant one of the plurality of groups arestored in said corresponding-image storage means; similarity evaluatingmeans (11) for evaluating a similarity in pose of each of the images ofthe relevant group stored in said corresponding-image storage means andfor the type specified by said type specifying means to the target imageextracted by said target image extracting unit; and means (11) forobtaining from said corresponding-image storage means an image for thespecified type and bearing a specified similarity in pose to the targetimage evaluated by said similarity evaluating means.
 8. The compositeimage output apparatus according to claim 1, characterized in that saidcorresponding-image obtaining unit comprises: corresponding-imagestorage means (12 d, 12 e, 12 f) having stored a plurality of modelimages and a plurality of groups of different images for a plurality ofrespective different types in corresponding relationship, each group ofimages corresponding to a respective one of the plurality of modelimages; type specifying means (11) for specifying any particular one ofthe plurality of different types for which different images of arelevant one of the plurality of groups are stored in saidcorresponding-image storage means; similarity evaluating means (11) forevaluating a similarity in pose of each of the model images stored insaid corresponding-image storage means to the target image extracted bysaid target image extracting unit; and means (11) for obtaining fromsaid corresponding image storage means an image for the type specifiedby said type specifying means, the image corresponding to a model imagebearing a specified pose similarity evaluated by said similarityevaluating means to the target image.
 9. The composite image outputapparatus according to claim 6, characterized by further comprising:means for adjusting each of the plurality of model images stored in saidcorresponding-image storage means in height and width to the targetimage extracted by the target image extracting unit, and characterizedin that: said similarity evaluating means evaluates a similarity in poseof each of the model images adjusted in height and width by said modelimage adjusting means to the target image extracted by said target imageextracting means.
 10. The composite image output apparatus according toclaim 1, characterized in that: said outputting unit adds thecorresponding-image obtained by said corresponding image obtaining unitto the picked-up scene image acquired by said picked-up scene imageacquiring unit at a position other than the position of the target imagein the picked-up scene image extracted by said target image extractingunit and outputs a resulting composite image.
 11. The composite imageoutput apparatus according to claim 1, characterized by furthercomprising: means (11, 16) for specifying movement of thecorresponding-image in the composite image outputted by said outputtingunit from its present position to a different position; and means (11)responsive to said specifying means for moving the corresponding-imageadded to the picked-up scene image from its present position to thespecified different position in the acquired scene image to therebyoutput a resulting composite image.
 12. The composite image outputapparatus according to claim 1, characterized in that said outputtingunit adds the corresponding-image obtained by said corresponding-imageobtaining unit to the latest one of the series of picked-up scene imagesacquired by said picked-up scene image acquiring unit at a positioncorresponding to the position of the target image extracted by saidtarget image extracting unit from among an older one of the series ofpicked-up scene images to thereby to output a resulting composite image.13. The composite image output apparatus according to claim 1,characterized in that said outputting unit mirror-reverses thecorresponding-image obtained by said corresponding-image obtaining unitand adds the mirror-reversed corresponding-image to the acquired sceneimage at a position where the mirror-reversed corresponding-image linkswith the object image in the picked-up scene image acquired by saidpicked-up scene image acquiring unit to thereby output a resultingcomposite image.
 14. The composite image output apparatus according toclaim 2, characterized in that said image outputting unit comprises: atarget image specifier (16) which specifies one from among a pluralityof candidates for the target image; and means (11, 20) for replacing theimage specified by said target image specifier with thecorresponding-image obtained by said corresponding-image obtaining unitin the picked-up scene image acquired by said picked-up scene imageacquiring unit to thereby output a resulting composite image.
 15. Thecomposite image output apparatus according to claim 2, characterized inthat said image outputting unit comprises: a target image specifier (16)which specifies one from a plurality of candidates for target imagesextracted by said target image extracting unit; and means (11, 20) foradding the image obtained by said corresponding-image obtaining unit tothe picked-up scene image acquired by said picked-up scene imageacquiring unit at the position of the target image specified by saidtarget image specifier, and the candidates for the remaining targetimages at the positions of the target images other than the target imagespecified by said target image specifier in decreasing order of theimage pickup distances to the respective objects to thereby output aresulting composite image.
 16. The composite image output apparatusaccording to claim 1, characterized in that said target image extractingunit extracts a target image from a first picked-up scene image acquiredby said picked-up scene image acquiring unit; and said outputting unitadds the image obtained by said corresponding-image obtaining unit to asecond picked-up scene image acquired by said picked-up image acquiringunit to thereby output a resulting composite image.
 17. The compositeimage output apparatus according to claim 1, characterized in that saidpicked-up scene image acquiring unit picks up a scene image.
 18. Thecomposite image output apparatus according to claim 1, characterized inthat said picked-up scene image acquiring unit acquires a picked-upscene image based upon a video image.
 19. The composite image outputapparatus according to claim 1, characterized in that the picked-upscene image comprises an animation mage.
 20. A recording medium havingrecorded a composite image outputting program that controls a computerso as to output a composite image, said program causing the computer tofunction as: picked-up scene image acquiring means for acquiring apicked-up scene image representing a scene containing an object; imageextracting means for extracting a target image representing the objectfrom the picked-up scene image acquired by said acquiring means;corresponding-image obtaining means for obtaining an image correspondingto the target image extracted by said extracting means; and imageoutputting means for adding the corresponding-image obtained by saidobtaining means to the picked-up scene image acquired by said picked-upscene image acquiring means and for outputting a resulting compositeimage.
 21. A recording medium having recorded a composite imageoutputting program that controls a computer so as to output a compositeimage, said program causing the computer to function as: picked-up sceneimage acquiring means for acquiring a plurality of picked-up sceneimages, each representing a scene containing an object; target imageextracting means for extracting a target image representing the objectfrom any selected one of the plurality of picked-up scene imagesacquired by said picked-up scene image acquiring means;corresponding-image obtaining means for obtaining an image correspondingto the target image extracted by said extracting means; and imageoutputting means for replacing in the picked-up scene image the targetimage extracted by said target image extracting means with thecorresponding-image acquired by said corresponding-image obtainingmeans, for adding the corresponding-image to any selected one of theothers of said plurality of picked-up scene images from which no targetimages are extracted by said target image extracting means, and foroutputting a series of resulting composite images.
 22. A composite imageoutput method comprising the steps of: acquiring a picked-up scene imagerepresenting a scene containing an object; extracting a target imagerepresenting the object from the picked-up scene image acquired by saidacquiring step; obtaining an image corresponding to the target imageextracted by said extracting step; and adding the corresponding-imageobtained by said obtaining step to the picked-up scene image acquired bysaid picked-up scene image acquiring step and for outputting a resultingcomposite image.
 23. A composite image output method comprising thesteps of: acquiring a plurality of picked-up scene images, eachrepresenting a scene containing an object; extracting a target imagerepresenting the object from any selected one of the plurality ofpicked-up scene images acquired by said picked-up scene image acquiringstep; obtaining an image corresponding to the target image extracted bysaid extracting step; and replacing in the picked-up scene image thetarget image extracted by said target image extracting step with thecorresponding-image acquired by said corresponding-image obtaining step,for adding the corresponding-image to any selected one of the others ofthe plurality of picked-up scene images from which no target images areextracted by said target image extracting step, and for outputting aresulting composite image.
 24. A composite image delivery apparatuscomprising: a picked-up scene image receiver (115, 116) which receivesfrom a terminal a picked-up scene image representing a scene containingan object; a target image extractor (111) which extracts a target imagerepresenting the object from the picked-up scene image received by saidpicked-up scene image receiver; a corresponding-image obtainer (111)which obtains an image corresponding to the target image extracted bysaid target image extractor; an image adder (111) which adds thecorresponding-image obtained by said corresponding-image obtainer to thepicked-up scene image to thereby provide a resulting composite image;and an image deliverer (115, 116) which delivers the composite imageprovided by said image adder to the terminal.
 25. The composite imagedelivery apparatus according to claim 24, wherein said picked-up sceneimage receiver receives a series of successively changing picked-upscene images, each image representing a scene containing an object, andwherein said target image extractor extracts a series of successivelychanging target images, each representing an act of the object, from theseries of picked-up scene images received by said picked-up scene imagereceiver.
 26. The composite image delivery apparatus according to claim24, wherein said corresponding-image obtainer comprises:corresponding-image storing means (112 d, 112 e, 112 f) having stored aplurality of different images, each corresponding to the target image;similarity evaluating means (111) for evaluating a similarity in pose ofeach of the different images stored in said corresponding-image storingmeans to the target image extracted by said target image extractingmeans; and means for acquiring a corresponding-image bearing a specifiedsimilarity in pose to the target image evaluated by said similarityevaluating means.
 27. The composite image delivery apparatus accordingto claim 26, wherein said similarity evaluator comprises:corresponding-image storage means (112 b, 112 c,112 d, 112 e, 112 f)having stored a plurality of model images and a plurality ofcorresponding images in corresponding relationship; and model similarityevaluating means (111) for evaluating a similarity in pose of each ofthe model images stored in said corresponding-image storage means to thetarget image extracted by said target image extracting means; andwherein said corresponding-image obtainer obtains from saidcorresponding image storage means an image corresponding to a modelimage bearing a specified similarity in pose to the target imageevaluated by said similarity evaluator.
 28. The composite image deliveryapparatus according to claim 24, further comprising: an address receiver(111, 115, 116) which receives from said terminal an address of adifferent terminal; and a deliverer (115, 116) which delivers thecomposite image provided by said image adder to the different terminalwhose address is received by said address receiver.
 29. A compositeimage delivery apparatus that communicates with a terminal, saidterminal extracting a target image contained in a picked-up scene image,sending said composite image delivery apparatus pose information on theextracted target image, receiving an image corresponding to the poseinformation that said composite image delivery apparatus sent inresponse to the sending of the pose information by said terminal, addingthe corresponding image to the picked-up scene image to thereby output aresulting composite image, said composite image delivery apparatuscomprising: a pose information receiver (115, 116) which receives fromsaid terminal the pose information on the target image contained in thepicked-up scene image; a corresponding-image storing memory (112 d, 112e, 112 f) having stored a plurality of images, each corresponding to arespective one of different items of pose information; and an imagedeliverer (111, 115, 116) which reads from said corresponding-imagestoring memory an image corresponding to the pose information receivedby said pose information receiver, and delivers the read image to saidterminal.
 30. A composite image delivery apparatus that communicateswith a terminal, said terminal extracting a target image contained in apicked-up scene image, sending said composite image delivery apparatuspose information on the extracted target image and information on thetype of a specified character, receiving an image corresponding to thepose information and the information on the type of a specifiedcharacter that said composite image delivery apparatus sent in responseto the sending of the pose information and the information on the typeof a specified character by said terminal, adding the received image tothe picked-up scene image to thereby output a resulting composite image,said composite image delivery apparatus comprising: a receiver (115,116) which receives from said terminal the pose information on thetarget image contained in the picked-up scene image and the informationon the type of a specified character; a corresponding-image storingmemory (112 d, 112 e, 112 f) having stored a plurality types of images,each type of images corresponding to a respective one of different itemsof pose information, each image representing a character; and an imagedeliverer (111, 115, 116) which reads from said corresponding-imagestoring memory an image corresponding to the pose information and theinformation on the type of a specified character received by said poseinformation receiver, and delivers the read image to the terminal. 31.The composite image delivery apparatus according to claim 30, furthercomprising: an position information receiver which receives from theterminal information on its position, and wherein: when the informationon the position of the terminal received by said position informationreceiver indicates that the terminal is at a specified location, saidimage deliverer reads from said corresponding-image storing memory animage representing the character and corresponding to the specifiedlocation and the pose information received by said pose informationreceiver, and sends the read image to the terminal.
 32. The compositeimage delivery apparatus according to claim 30, wherein: said imagedeliverer reads from said corresponding-image storing memory an imagerepresenting the character and corresponding to the pose informationreceived by said pose information receiving memory and information onthe particular time zone when the present time is in a particular timezone, and delivers the read image to the terminal.
 33. The compositeimage delivery apparatus according to claim 24, further comprising asender which sends the terminal a bill for the creating and deliversservices of the image provided by said deliverer.
 34. A recording mediumhaving stored a composite image delivery program for controlling acomputer connected to a network to deliver a composite image to aterminal, said program causing the computer to function as: picked-upscene image receiving means for receiving from the terminal a picked-upscene image representing a scene containing an object; target imageextracting means for extracting a target image representing the objectfrom the picked-up scene image received by said picked-up scene imagereceiving means; corresponding-image obtaining means for obtaining animage corresponding to the target image extracted by said target imageextracting means; image adding means for adding the image obtained bysaid corresponding-image obtaining means to the picked-up scene image tothereby provide a resulting composite image; and image delivering meansfor delivering to the terminal the composite image provided by saidimage adding means.
 35. A composite image delivering method comprisingthe steps of: receiving from a terminal a picked-up scene imagerepresenting a scene containing an object; extracting a target imagerepresenting the object from the picked-up scene image received by saidreceiving step; obtaining an image corresponding to the target imageextracted by said extracting step; adding the image obtained by saidcorresponding-image obtaining step to the picked-up scene image tothereby provide a resulting composite image; and delivering to theterminal the composite image provided by said image adding step.
 36. Arecording medium having stored a composite image delivery program thatcontrols a computer communicating with a terminal, said terminalextracting a target image contained in a picked-up scene image, thetarget image corresponding to an object, sending an image deliveryapparatus pose information on the extracted target image, receiving animage corresponding to the pose information that the image deliveryapparatus sent in response to the sending of the pose information by theterminal, and adding the corresponding image to the picked-up sceneimage to thereby output a resulting composite image, said programcausing said computer to function as: pose information receiving meansfor receiving from said terminal pose information on the target imagecorresponding to the object image and contained in the picked-up sceneimage; corresponding-image storing means having stored a plurality ofimages, each corresponding to a respective one of different items ofpose information; and image delivering means for reading from saidcorresponding-image storing means an image corresponding to the poseinformation received by said pose information receiving means, and fordelivering the read image to the terminal.
 37. A composite imagedelivering method communicating with a terminal, the terminal extractinga target image contained in a picked-up scene image, the target imagecorresponding to an object, sending an image delivery apparatus poseinformation on the extracted target image, receiving an imagecorresponding to the pose information that the image delivery apparatussent in response to the sending of the pose information by the terminal,and adding the corresponding image to the picked-up scene image tothereby output a resulting composite image, said method comprising thesteps of: receiving from the terminal the pose information on the targetimage contained in the picked-up scene image; storing in a memory aplurality of images, each corresponding to a respective one of differentitems of pose information; and reading from the memory an imagecorresponding to the pose information received by said pose informationreceiving step, and for delivering the read image to the terminal. 38.The composite image delivery apparatus according to claim 29, whereinthe images stored in said corresponding-image storing memory includesthe images extracted from the picked-up scene image in the terminal andreceived from the terminal, the images being stored in correspondence toitems of their pose information.
 39. The composite image deliveryapparatus according to claim 29, wherein the images stored in saidcorresponding-image storing memory comprise images downloaded by theterminal from the Internet and stored in correspondence to items oftheir pose information.